-- Logs begin at Mon 2022-05-23 19:26:58 UTC, end at Mon 2022-05-23 20:40:14 UTC. -- May 23 20:03:15.655112 dv-cinder-843018-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 20:03:15.663359 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 20:03:15 2022] *** May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: nodename: dv-cinder-843018-1-iscsi May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: machine: x86_64 May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: clock source: unix May 23 20:03:15.665364 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: pcre jit disabled May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: detected number of CPU cores: 4 May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: current working directory: / May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: detected binary path: /usr/bin/uwsgi-core May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: your processes number limit is 47680 May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: your memory page size is 4096 bytes May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: detected max file descriptor number: 2048 May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: lock engine: pthread robust mutexes May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: thunder lock: enabled May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 20:03:15.665824 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 20:03:15.716952 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: Python main interpreter initialized at 0x56179de6f9c0 May 23 20:03:15.716952 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: python threads support enabled May 23 20:03:15.716952 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: your server socket listen backlog is limited to 100 connections May 23 20:03:15.716952 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: your mercy for graceful operations on workers is 90 seconds May 23 20:03:15.717326 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: mapped 403077 bytes (393 KB) for 2 cores May 23 20:03:15.717403 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: *** Operational MODE: preforking *** May 23 20:03:15.717682 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: *** uWSGI is running in multiple interpreter mode *** May 23 20:03:15.717682 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: spawned uWSGI master process (pid: 80591) May 23 20:03:15.717883 dv-cinder-843018-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 20:03:15.719686 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: spawned uWSGI worker 1 (pid: 80603, cores: 1) May 23 20:03:15.719686 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: spawned uWSGI worker 2 (pid: 80604, cores: 1) May 23 20:03:15.719686 dv-cinder-843018-1-iscsi devstack@keystone.service[80591]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 20:03:21.376095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 20:03:21.376095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: from cryptography.utils import int_from_bytes May 23 20:03:21.376859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 20:03:21.376859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: from cryptography.utils import int_from_bytes May 23 20:03:21.377283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 20:03:21.377283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: from cryptography.utils import int_from_bytes May 23 20:03:21.377410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 20:03:21.377410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: from cryptography.utils import int_from_bytes May 23 20:03:21.425818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 20:03:21.425968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 20:03:21.426089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 20:03:21.426180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 20:03:21.426252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] command line args: [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 20:03:21.426320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] command line args: [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 20:03:21.426465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 20:03:21.426548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 20:03:21.426646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ================================================================================ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 20:03:21.426724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ================================================================================ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 20:03:21.426900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] admin_token = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.426983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] admin_token = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_dir = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_dir = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_source = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_source = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] debug = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] debug = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.427867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.428933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_config_append = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_config_append = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_dir = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_dir = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.429996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.430942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.431968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.432919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] publish_errors = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.433921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] publish_errors = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.434976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] standard_threads = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] standard_threads = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.435997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_journal = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_journal = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_json = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_json = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_stderr = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_stderr = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_syslog = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_syslog = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 20:03:21.436973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.437956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438539 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.438940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.439983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.440916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.441885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.442923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.443836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.444951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.445904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.446924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.447860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.448930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.449953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.450914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.451848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.452562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.453977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.454957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.455903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.456873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.457962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.458936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.459932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.460848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.461950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.external = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.462930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.external = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.password = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.token = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.463941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.password = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.token = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.464901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.465989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.466965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.467947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.468024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.468184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.468361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.469900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.470889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.471999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.472951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.473986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.474955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.475893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.476847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.477845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.478809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.479958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.480928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.481983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.482912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.483855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.484968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.485978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.486966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487353 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.487825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.488867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.489972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.490933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.491952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.492900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.493852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.494981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.495895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.496855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.497974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.498966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.499957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.500911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.501993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.502859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.503985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.504996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.505943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.506927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.507872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.508991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.509964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.510958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.511996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.512937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.513970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 20:03:21.514065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.514293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.514526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.514767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.515004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 20:03:21.515706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 20:03:21.545750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.545899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.546083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.546213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.546295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.554861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.554982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.583757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.583911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.583997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.584145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.637383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.637613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.689843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.690094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.690549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.690784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.756418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.756592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.756816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.756928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.757939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.781918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.782970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.793979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.794288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.806607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.806781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.806864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.807956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.808088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 20:03:21.827515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.828260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 20:03:21.828348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.828562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 20:03:21.828641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.828790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 20:03:21.828894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 20:03:21.829311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.829896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.830101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.840917 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.841912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.849846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.850045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.853126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.853312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.853396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.853593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.853674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.853874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.854970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.855966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.856149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.856226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.856416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.882882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.883288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.884595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.884873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.885019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.885297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.890878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.890964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.891200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.891283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.891513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.891791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.898454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.898744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.899095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.899376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.903212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.903613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.903857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.903938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.904983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.905072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.905279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.905373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.905552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.905845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.923817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.924092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.924441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.924717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.928752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.929374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.932400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.932695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.933035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.933328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.938256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.938911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.941860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.942532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.945649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.945947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.946913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.947098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.947181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.947412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.947688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.966631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.966914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.967178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.967261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.967438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.967520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.967703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.967964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.975419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.976014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.977132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.977407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.977725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.978000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.980744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.981024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.981304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.981385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.981587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.981668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.981865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.981947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.982163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.982439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.995640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.995814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:21.995909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.996742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.997050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.997274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.997392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.997480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:21.997544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.016371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.016537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.019690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.019824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.023001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.023158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.024620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.024760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.026267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.026437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.026556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.026727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.026834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.027007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.034222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.034399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.035826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.036005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.039150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.039255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.039443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.039526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.039730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.039812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.040002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.040084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.049601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.049693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.049886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.049979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.050293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.050376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.058169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.058252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 20:03:22.058451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.058533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.058728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.058809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.059916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.060983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.061765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 20:03:22.100552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56179de6f9c0 pid: 80604 (default app) May 23 20:03:22.100728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56179de6f9c0 pid: 80603 (default app) May 23 20:03:22.103774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160085-896a-47bc-8aed-be4890e2427d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:22.104337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160085-896a-47bc-8aed-be4890e2427d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:22.104614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce160085-896a-47bc-8aed-be4890e2427d None None] PATH_INFO: `/v3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:22.109420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Mon May 23 20:03:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:31.067557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:31.068116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:31.068116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:31.072589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:31.077850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:31.078336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:31.078589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:31.146443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: INFO dbcounter [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] Registered counter for database keystone May 23 20:03:31.167109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG oslo_db.sqlalchemy.engines [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 20:03:31.173832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writer thread running {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 20:03:31.346091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG passlib.handlers.bcrypt [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 20:03:31.346454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG passlib.handlers.bcrypt [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 20:03:31.401656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:31.403265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:31.523361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 23 20:03:31.729143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:31.798000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: INFO dbcounter [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] Registered counter for database keystone May 23 20:03:31.816651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG oslo_db.sqlalchemy.engines [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 20:03:31.822760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writer thread running {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 20:03:31.941184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5e01c-25c8-48f7-9121-9fbbf3722080 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:31.945655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:31.946333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:31.946616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:31.946888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:31.955835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:32.007113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING keystone.common.rbac_enforcer.enforcer [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 20:03:32.007377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c976f-e6e9-407d-baba-963cb86dee93 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:32.017345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Mon May 23 20:03:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 20:03:32.023325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:32.038313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bcf48-84eb-40f0-83e4-0ac813fbf0b5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:32.042665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:32.043769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:32.044056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:32.044337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] PATH_INFO: `/v3/projects/f5277fd37f8044ad9345367a8cd3028d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:32.045824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] RBAC: Authorizing `identity:get_project(project_id=f5277fd37f8044ad9345367a8cd3028d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:32.203027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 20:03:32.203671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35cde0-382e-494e-8a21-18c2578cc5d3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:32.208555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Mon May 23 20:03:32 2022] GET /identity/v3/projects/f5277fd37f8044ad9345367a8cd3028d => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:34.353230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7207494-add8-4e02-96f4-3ae1b2aaf922 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:34.354063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7207494-add8-4e02-96f4-3ae1b2aaf922 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:34.354063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7207494-add8-4e02-96f4-3ae1b2aaf922 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:34.358185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:03:34.364184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:34.364558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:34.364751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:34.383261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:34.384770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:34.392532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 20:03:34.414532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7207494-add8-4e02-96f4-3ae1b2aaf922 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:34.426850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7207494-add8-4e02-96f4-3ae1b2aaf922 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:34.430993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:34.431617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:34.431902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:34.432182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:34.439752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:34.441517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c2897-426a-4b06-a75d-28a2875a831c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:34.450165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Mon May 23 20:03:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:03:34.455343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:34.464095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04a57-16ab-4583-ac24-272c7629dfee None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:34.468454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:34.469306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:34.470152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:34.470428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:34.471811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:34.473684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e87c9-c113-449a-9e79-223851628137 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:34.475202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Mon May 23 20:03:34 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:36.697932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41fb1-bffc-4b5e-99f3-558cbca4f865 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:36.698504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41fb1-bffc-4b5e-99f3-558cbca4f865 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:36.698504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41fb1-bffc-4b5e-99f3-558cbca4f865 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:36.699868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:36.705679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:36.706132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:36.706356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:36.722593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:36.723765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:36.744712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed39-c8e9-4c0c-9785-ea9442765895 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:36.745002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed39-c8e9-4c0c-9785-ea9442765895 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:36.745276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed39-c8e9-4c0c-9785-ea9442765895 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:36.746639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:03:36.752100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:36.752380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:36.752654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:36.792162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG passlib.handlers.bcrypt [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80603) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 20:03:36.792584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG passlib.handlers.bcrypt [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80603) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 20:03:36.802572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:36.825530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:36.839682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:36.841281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:36.848572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:36.929078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1604be-8f11-46bc-a20d-57bca33f32a0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:36.940117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:36.962797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:36.964516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:36.965310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:36.966347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:36.966347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:36.978088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:36.990457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:36.991696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:36.997421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:03:36.998138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.083439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.101118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db524f6d-247f-4884-844d-4dc36231c25f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.138351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.139009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.139289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.139561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.140309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:37.140611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.141996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.153308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:37.158909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.170172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a67c76-1730-468a-ac39-4dcb333a0b36 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.173692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.213557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.214463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.214755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.215033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.260265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44c342-5c63-49a9-9d29-13125e191910 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.281412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.282952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.293303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.294204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.294488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.294764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.295973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.297434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.299586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 9/16] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:37 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:37.304952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.319094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.382314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 8/17] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:03:37.387823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.390530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.401290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab120a98-3efc-4d64-8b51-e515ce0a7033 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.435110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.435900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.436183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.436459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.437634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.439154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.440923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:37.446420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.483731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f590a-aea3-4ece-a673-7705cb070b4d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.516691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.517484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.517769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.518065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.524249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.537304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.538799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.542714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5bf4-b6f0-4e94-8c1a-004b84d7b4d1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:03:37.559943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:37.560822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:37.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:03:37.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:03:37.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:03:37.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:03:37.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:03:37.567048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:37.581793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.582589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.582874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.583145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.583977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.domain.id=default, user.id=a03bfabd375944c7923bb6e092aa3498) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:37.584285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.585734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13bf43-168c-404b-97ce-610345d9a459 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.592263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Mon May 23 20:03:37 2022] GET /identity/v3/role_assignments?user.id=a03bfabd375944c7923bb6e092aa3498&scope.domain.id=default&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:37.625397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:37.670386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4bf7c-bf2b-4a0d-956f-14c5dfca87b3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:37.706052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:37.706749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:37.707024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:37.707304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:37.708052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:37.708362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:37.709772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9634-6936-4de6-b1ec-b03f38acf0ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:37.729339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:39.904745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a42b9-3079-4f31-8c06-6ae5d0122fdd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:39.905753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a42b9-3079-4f31-8c06-6ae5d0122fdd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:39.906270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a42b9-3079-4f31-8c06-6ae5d0122fdd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:39.907888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:39.913477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:39.914349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:39.914784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:39.936752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:39.938823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:40.031480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:40.067888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a42b9-3079-4f31-8c06-6ae5d0122fdd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:40.206800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a42b9-3079-4f31-8c06-6ae5d0122fdd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:40.260586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:40.261422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.261704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.261979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.282694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:40.290626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:40.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:40.292266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:40.296578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 13/24] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:40.298335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:40.389753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:40.475903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ad92-e5a0-4f39-b5f6-642c4a88f79b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:40.527683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:40.528722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.529205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.529684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.530654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:40.531182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:40.533285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5b6f5-6794-470c-bf3d-eb70bc1b4255 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:40.553254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 12/25] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:40.559498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:40.571236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:40.577236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7b8eb-1ba1-493d-8d75-43e24aff8aab None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.577845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7b8eb-1ba1-493d-8d75-43e24aff8aab None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.578384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7b8eb-1ba1-493d-8d75-43e24aff8aab None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.580145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:03:40.586061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.586695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.587183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.614357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:40.618497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:40.717289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69c30-ca49-4ebf-acc2-0f7529258769 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:40.756615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:40.757812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.758309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.758892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.760442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:40.762110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:40.764056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 14/27] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:40 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:40.770292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:40.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:40.821271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:40.869031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:40.874454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:40.905340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa3a5-2514-4e2d-97a5-b26841a0d6c9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:40.950079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c36f6-e55f-461a-9697-610522e4c680 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:40.951677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:40.954240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.958388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.958889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.961595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:40.964761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e443-c1b7-49f9-801b-7a86d73a9a45 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:40.967029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 15/29] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:40.969079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43a8b-8801-43c2-9097-868b808878f0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.969920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43a8b-8801-43c2-9097-868b808878f0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.970228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd43a8b-8801-43c2-9097-868b808878f0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.974060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 16/30] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:40.974060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ad3-2f6e-48d7-9f14-0e3e25c44a8d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:40.974060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ad3-2f6e-48d7-9f14-0e3e25c44a8d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:40.974060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7719ad3-2f6e-48d7-9f14-0e3e25c44a8d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:40.975271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 17/31] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:40.976210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7719ad3-2f6e-48d7-9f14-0e3e25c44a8d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:41.000841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:41.001868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.002704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.003078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.003983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:41.005985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5d326-e4e1-432c-ab28-368b0b402185 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:41.088793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7719ad3-2f6e-48d7-9f14-0e3e25c44a8d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:41.127675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:41.128647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.128933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.129214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] PATH_INFO: `/v3/domains/default/users/a03bfabd375944c7923bb6e092aa3498/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.131397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a03bfabd375944c7923bb6e092aa3498, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:41.132974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f1688-2ba0-4b68-8443-cd6867af1500 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:41.221775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 15/32] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 20:03:40 2022] POST /identity/v3/roles => generated 219 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:03:41.223272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.223619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.223880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.243240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:41.267975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:41.273044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:41.416738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 18/33] 10.222.0.16 () {58 vars in 1361 bytes} [Mon May 23 20:03:40 2022] PUT /identity/v3/domains/default/users/a03bfabd375944c7923bb6e092aa3498/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:03:41.418836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.419268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.420370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.441712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:41.443550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:41.466781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 16/34] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:41.519245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:41.526448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 19/35] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:41.557357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:41.575148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:41.684033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cc24c-4309-4e4e-9a54-0f6d26d3854f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:41.699061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c2cc-2319-414a-a6c2-d77cac416fdb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:41.718644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:41.724984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:41.725818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.726145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.726423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.738680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:41.740472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:03:41.761411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:41.762286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:41.763077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:03:41.763077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:03:41.763077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:03:41.763077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:41.763077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 20/36] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:03:41.768773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:41.782314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:41.783021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.783344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.783703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.784481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:41.786125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8d794-a2ec-44b9-9461-43e2d9145c54 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:41.876584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 17/37] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 20:03:41 2022] POST /identity/v3/roles => generated 225 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:03:41.888246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80282b-2ead-4779-9659-84e8527e19ac None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:41.923961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7fa0a-14a0-47ac-960b-9b8a87272156 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.924410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7fa0a-14a0-47ac-960b-9b8a87272156 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.924642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7fa0a-14a0-47ac-960b-9b8a87272156 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.926295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 18/38] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:03:41.932592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655aca-b1cc-42db-9cee-fa0806c783fe None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.932986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655aca-b1cc-42db-9cee-fa0806c783fe None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.933282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655aca-b1cc-42db-9cee-fa0806c783fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.933410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:41.934207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:41.934489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:41.934830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:41.935796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:41.936162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:41.937846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:41.953212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-98655aca-b1cc-42db-9cee-fa0806c783fe None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:41.954666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-98655aca-b1cc-42db-9cee-fa0806c783fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:41.960564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 21/39] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:41.967033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:42.025738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:42.060069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 19/40] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:42.061502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd6f7-f15c-4e7c-a5d0-c7e74d3a1659 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.061851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd6f7-f15c-4e7c-a5d0-c7e74d3a1659 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.062168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dd6f7-f15c-4e7c-a5d0-c7e74d3a1659 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.063554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 20/41] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:42.069026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.069325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.069614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.088336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:42.089731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:42.096645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5ed9-7e6b-438e-9f08-1808fe458794 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:42.105867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:42.136226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:42.136978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.137649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.137649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.138308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:42.140019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:42.169550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 21/42] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:42.170578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:42.268034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 22/43] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:03:41 2022] POST /identity/v3/projects => generated 301 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:03:42.269061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:42.290211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893e70d-8c03-4e6e-a665-db48c50e6345 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:42.331159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:42.332391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.332888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.333373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.334406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:42.336174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:42.379638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:42.387617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24de2b-1553-4b50-9861-33ed6e4cc984 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:42.412095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 22/44] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 20:03:42 2022] POST /identity/v3/roles => generated 223 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:42.432734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:42.434046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.434608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.435119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.436891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:42.438758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b69e-83a3-46e8-9f8a-2b7a044a3d9e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:42.441047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 23/45] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:42.447058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:42.461216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:42.566950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8119e41-03eb-4e9b-bb7b-4345a6963c7c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:42.616198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:42.616880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:42.617173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:42.617455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:42.618625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:42.620124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd908-6bcc-462e-aa75-6ebcc37a1b0d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:42.713762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:03:42 2022] POST /identity/v3/projects => generated 312 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:44.006435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab76ae-db55-4e5a-b098-2b9af7097e13 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.006945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab76ae-db55-4e5a-b098-2b9af7097e13 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.007049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab76ae-db55-4e5a-b098-2b9af7097e13 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.008360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:44.013464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.013851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.014183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.031040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:44.032220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:44.085695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:44.096864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:44.121174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab76ae-db55-4e5a-b098-2b9af7097e13 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:44.220912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab76ae-db55-4e5a-b098-2b9af7097e13 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:44.254243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:44.255036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.255318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.255595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.265302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:44.272864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:44.273727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:44.274763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:44.279791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:44.376901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005d0a7-2dbd-4cbe-9937-da153e207098 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:44.409957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:44.410657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.410943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.411219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.412293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:44.412613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:44.414007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:44.422009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 25/50] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:44.429425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:44.453292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:44.527136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20219a-125f-4c77-9ec9-0eb753ba0986 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:44.559427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:44.560297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.560581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.560853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.562057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:44.563507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:44.565121 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:44 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:44.570548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:44.621846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:44.667211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784021b2-fe97-45c5-84bd-0da054c6954d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:44.698959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:44.699855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.700152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.700428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.701595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:44.703064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642676d3-1f6a-4906-8622-b25fb9f57417 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:44.705188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 26/52] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:44.710907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:44.802111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:44.811173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a394643-b9fa-4ba2-bc55-84ba2e10f4b9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:44.845046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:44.845690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.845972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.846270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.847100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.domain.id=default, user.id=a03bfabd375944c7923bb6e092aa3498) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:44.847409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:44.849030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc0b04-4710-4784-95b9-e03ca06d09ef None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:44.854880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 27/53] 10.222.0.16 () {56 vars in 1322 bytes} [Mon May 23 20:03:44 2022] GET /identity/v3/role_assignments?user.id=a03bfabd375944c7923bb6e092aa3498&scope.domain.id=default&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:44.916410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243043e-1206-4844-8661-07e14fd1baa8 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.916844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243043e-1206-4844-8661-07e14fd1baa8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.916991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243043e-1206-4844-8661-07e14fd1baa8 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.918464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 27/54] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:44.924446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:44.924740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:44.925022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:44.943177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:44.944420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:45.017021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 28/55] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:45.040271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243043e-1206-4844-8661-07e14fd1baa8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:45.143943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243043e-1206-4844-8661-07e14fd1baa8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:45.162673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:45.175642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:45.176439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:45.176725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:45.177002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:45.178204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:45.179640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddb603-0609-41fa-9fcd-770699a59dd0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:45.181240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 28/56] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:45.186900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:45.268449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:45.282980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcfc16-a073-4223-b805-03a314763b8e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:45.316232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:45.316892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:45.317177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:45.317454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:45.318188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:45.319643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cdddf-619b-4880-88c2-9cf974d13039 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:45.400445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 29/57] 10.222.0.16 () {60 vars in 1140 bytes} [Mon May 23 20:03:45 2022] POST /identity/v3/projects => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:47.571918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979438-bc2b-417e-b06e-c855d6362935 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:47.572425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979438-bc2b-417e-b06e-c855d6362935 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:47.572425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86979438-bc2b-417e-b06e-c855d6362935 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:47.573776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 29/58] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:47.579516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:47.579900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:47.580069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:47.597038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:47.598268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:47.662341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 30/59] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:47.685946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979438-bc2b-417e-b06e-c855d6362935 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:47.783061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86979438-bc2b-417e-b06e-c855d6362935 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:47.814409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:47.815186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:47.815471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:47.815746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:47.816914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:47.818378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14388ec4-14a9-4331-8e44-d120f2b18c0e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:47.819945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 30/60] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:47.825103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:47.889467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:47.925257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f1bb4-8641-48ce-a371-86f44c62d200 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:47.966224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:47.966904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:47.967190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:47.967468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:47.968187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:47.969963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77007ad3-9888-43e7-bd35-50514e389a27 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:48.074261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 31/61] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:03:47 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:51.423064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da005e77-d1d9-471e-bb94-9c4090cba86a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.424081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da005e77-d1d9-471e-bb94-9c4090cba86a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.424564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da005e77-d1d9-471e-bb94-9c4090cba86a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.426177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 31/62] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:51.431746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f5f9-a2c8-4890-b386-d3f58fcb7ae4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.436071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f5f9-a2c8-4890-b386-d3f58fcb7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.436071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f5f9-a2c8-4890-b386-d3f58fcb7ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.454674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3897f5f9-a2c8-4890-b386-d3f58fcb7ae4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:51.455877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3897f5f9-a2c8-4890-b386-d3f58fcb7ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:51.505557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dbc1-c0ac-48fb-b2d9-ceea7d6bd0f7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.506181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dbc1-c0ac-48fb-b2d9-ceea7d6bd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.506675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26dbc1-c0ac-48fb-b2d9-ceea7d6bd0f7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.510048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 32/63] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:51.516746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.518128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.520324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.553233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:51.555739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:51.562516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 32/64] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:51.564090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2b50-7251-4190-a3f8-320d777b5d72 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.564526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2b50-7251-4190-a3f8-320d777b5d72 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.564974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2b50-7251-4190-a3f8-320d777b5d72 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.566677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 33/65] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:51.571860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.572307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.572730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.598377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:51.599912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:51.615714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:51.633928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 33/66] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:51.634972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:51.662776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:51.693895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 34/67] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:51.695065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:51.739242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264d62-021f-4d02-8d31-aedf686452a0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:51.781738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:51.782929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.783454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.783948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.797071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:51.807210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:51.808066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:51.808828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:03:51.811653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 34/68] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:51.816219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70cc4-8874-4368-bc85-5b36012f4083 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:51.817897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:51.859976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:51.861091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.861928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.862412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.868877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:51.875126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:51.884050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:51.884954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:51.885694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:51.888349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 35/69] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:51.894980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:51.932550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ee0-6906-47ef-8763-fea2174f3873 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:51.941106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:51.969987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:51.970834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:51.971122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:51.971404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:51.982899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:51.991817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:51.992848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:03:51.993785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:03:51.995800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 35/70] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:03:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:51.996809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.007343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe0936-13f5-40e2-bda8-d59cc463ebde None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.044326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.044931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.045215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.045493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.046272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:52.046585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.048481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e25c3-7cf8-494d-8f0f-4288ba1b9255 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.056452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:03:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.057912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb43a4-7124-4d93-a457-65e7dfc361dc None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.058322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb43a4-7124-4d93-a457-65e7dfc361dc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.058606 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb43a4-7124-4d93-a457-65e7dfc361dc None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.060130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 37/72] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:52.065146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb43a4-7124-4d93-a457-65e7dfc361dc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.106540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68083591-75f8-4f89-bc65-413a64c3d74e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.113747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.139811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.140480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.140769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.141073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.141827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:52.142174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.143562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.151371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 36/73] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.152162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.166532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb43a4-7124-4d93-a457-65e7dfc361dc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.169544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.200894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.201556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.201839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.202145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.202890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:52.203196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.205106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3c184-4d22-45df-8e85-47e786fa3be4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.212938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 38/74] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.213954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3486c2-7dd7-4de6-ae50-b079cf803f10 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.214309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3486c2-7dd7-4de6-ae50-b079cf803f10 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.214591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3486c2-7dd7-4de6-ae50-b079cf803f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.231105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5d3486c2-7dd7-4de6-ae50-b079cf803f10 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:52.232302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5d3486c2-7dd7-4de6-ae50-b079cf803f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:52.257857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e8cd-6841-4aea-8d5e-c140e81838e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.290535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.291408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.291696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.291967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.296503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 39/75] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:52.297729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965746-c8d6-4bc5-9153-c7d7aeb0c9fe None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.298037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965746-c8d6-4bc5-9153-c7d7aeb0c9fe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.298328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965746-c8d6-4bc5-9153-c7d7aeb0c9fe None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.303787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 40/76] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:52.303787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965746-c8d6-4bc5-9153-c7d7aeb0c9fe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.311700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.313232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.319217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 37/77] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.319217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.342461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.402816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.409523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965746-c8d6-4bc5-9153-c7d7aeb0c9fe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.413192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe68726-f646-4a37-8c41-166e31fb5e01 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.442963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.443834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.444116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.444395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.445561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.446288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.447052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9ff3-e014-4439-88df-22dba78a3508 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.447184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.447451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.447729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.448685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 41/78] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.448931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.449796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.450106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.450390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.450475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.452022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 38/79] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:52.452777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.468165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:52.469352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:52.535592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 42/80] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:52.542925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.545679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.552079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81889ea9-9c05-48b5-b19b-ea3871d61112 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.585652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.586742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.587204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.587920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.597917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:52.605873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:52.606750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:03:52.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 39/81] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:52.608486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.635819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.642950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ea4b6-8172-40b6-9183-e85a923bf63b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.675577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.676223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.676511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.676790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.705669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4bf13-8a5c-47d2-9e0e-e4d49920ec95 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.738216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.738860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.739146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.739425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.741907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.743406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.762527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.805325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:52.806756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:52.831049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.837871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 43/82] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:03:52.838759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.900929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 40/83] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:03:52.901765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:52.939033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66edad-5cd1-462d-98e5-1a6e573b3d75 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:52.953705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:52.970561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:52.971207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:52.971494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:52.971773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:52.996570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246241e9-8f53-41a9-9f00-310c7e7421f0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.030383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.030887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.031166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.031444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.031752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.032914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.034376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.035970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 41/84] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.036806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.041618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.043129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.134132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93139-3a69-4b4a-bb78-a07f4cce3b17 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.139109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 44/85] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:03:53.139939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.159012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.170037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.170718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.171007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.171289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.178116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:53.178116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.178116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.183195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 42/86] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.183793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.237038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.243501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc359fb-7722-4a05-bd09-f43a709176c2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.275978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.276835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.277137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.277390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.278597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.279779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed1bb1-4383-44f9-88cb-e15c3a3dd415 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.280064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.282306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 45/87] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.283141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.313985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.314878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.315163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.315443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.316636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.318122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.319808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 43/88] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.320567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.372995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.382453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa2c7-da0e-4b68-8521-64f2a36447f8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.416114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.416718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.417008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.417281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.417986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.419461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.420268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af9f1-be93-47c2-a842-5476dadabbb2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.452470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.453352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.453655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.453932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.455151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.456580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.458306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 44/89] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.459060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.494690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 46/90] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:03:53 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:53.495515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.517730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.571814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c51089-0310-4273-bdd7-07feeedee39d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.596742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.612612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.613529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.613821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.614119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.615438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.616840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.617283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c41c2d-3639-4ca5-a2e0-d34a60e8b699 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.618817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 45/91] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.619402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.659564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.660245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.660534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.660816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.661668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:53.661986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.663486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.669945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 47/92] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.670766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.764108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d888-8d60-41dd-8bc7-969feb8e26af None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.802036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.817088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.818847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.819306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.819796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.820870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=a03bfabd375944c7923bb6e092aa3498) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:53.821417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.823164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.833559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de366-dbb2-42e2-ba4d-1bb9c82fe337 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:53.842133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 46/93] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/role_assignments?user.id=a03bfabd375944c7923bb6e092aa3498&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.842133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:53.905834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:53.906623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:53.906905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:53.907180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:53.908035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:53.908348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:53.910566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:53.919574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 48/94] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:53.925529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:53.988014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd72b6-babb-4812-96dd-ea94e50d2c51 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:54.028957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:54.029694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:54.029988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:54.030298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:54.115679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:54.117196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:54.176804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:54.233356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 47/95] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:03:54.239603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:54.375928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26ddc8-5e79-4fac-8eeb-2ae8c9a17068 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:54.410142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:54.411015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:54.411300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:54.411575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] PATH_INFO: `/v3/projects/e40e053c1881470fa8cbf538026212e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:54.412792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] RBAC: Authorizing `identity:get_project(project_id=e40e053c1881470fa8cbf538026212e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:54.414285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50065b1a-02a7-465b-af00-55ca31722977 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:54.416212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 49/96] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:54 2022] GET /identity/v3/projects/e40e053c1881470fa8cbf538026212e2 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:54.421769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:54.538688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67e10-2859-400c-a048-ec2e4c31e1e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:54.588719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:54.589388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:54.589673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:54.589950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:54.590818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=e40e053c1881470fa8cbf538026212e2, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:54.591134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:54.592598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e136b47-8aa3-465d-bcdf-fbd0cebe9006 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:54.598815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 48/97] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:03:54 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=e40e053c1881470fa8cbf538026212e2&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:56.258259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dea42-5cd3-4a2b-99b4-88fa3c607928 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:56.259088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dea42-5cd3-4a2b-99b4-88fa3c607928 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:56.259578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dea42-5cd3-4a2b-99b4-88fa3c607928 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:56.261166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 50/98] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:56.266719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:56.267507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:56.267986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:56.294995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:56.296530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:56.308499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:56.378761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 49/99] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:56.412904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dea42-5cd3-4a2b-99b4-88fa3c607928 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:56.434818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:56.542778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dea42-5cd3-4a2b-99b4-88fa3c607928 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:56.582463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:56.583626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:56.584125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:56.584604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:56.599419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:56.607625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:56.608466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:56.609216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:56.615731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 51/100] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:56.621291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:56.749654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deca05f-f325-43ed-9095-70f0094eda8c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:56.799058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:56.800034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:56.800467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:56.800889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:56.801816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:56.802302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:56.803898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3a438-3005-4c9a-9fc3-616763f7dc9a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:56.813787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 50/101] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:03:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:03:56.820162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.001169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.007276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48d29a-aade-4c7d-a49e-c15687e5ec5f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.007857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48d29a-aade-4c7d-a49e-c15687e5ec5f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.008362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48d29a-aade-4c7d-a49e-c15687e5ec5f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.009961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 51/102] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:03:57.015225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc86d-418b-4eba-b780-edd10905dbcb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.015823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc86d-418b-4eba-b780-edd10905dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.016789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc86d-418b-4eba-b780-edd10905dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.045627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6896-3cf4-4636-a772-af04476976c0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.046829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-58ebc86d-418b-4eba-b780-edd10905dbcb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:57.049257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-58ebc86d-418b-4eba-b780-edd10905dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:57.054040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.109138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.110089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.110372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.110645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.111870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.113341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f766a-5470-442c-86b6-50ca2238f8b4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.115317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 52/103] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:56 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:57.116981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6626bc-38ac-42ca-85fa-a01dc4f459c4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.116981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6626bc-38ac-42ca-85fa-a01dc4f459c4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.117145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6626bc-38ac-42ca-85fa-a01dc4f459c4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.118381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 53/104] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:57.122064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9534-c5c8-4bdb-bcd0-e04f71a56e5e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.122064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9534-c5c8-4bdb-bcd0-e04f71a56e5e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.122064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9534-c5c8-4bdb-bcd0-e04f71a56e5e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.122064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 54/105] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:57.122367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9534-c5c8-4bdb-bcd0-e04f71a56e5e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.161877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 52/106] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:57.163482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ce5a-29fe-4df4-8747-87ffd92d3644 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.163775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ce5a-29fe-4df4-8747-87ffd92d3644 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.164054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684ce5a-29fe-4df4-8747-87ffd92d3644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.193299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4684ce5a-29fe-4df4-8747-87ffd92d3644 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:57.194517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4684ce5a-29fe-4df4-8747-87ffd92d3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:57.243221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9534-c5c8-4bdb-bcd0-e04f71a56e5e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.268908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 53/107] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:57.275873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.276228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.276550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.279383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.280746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.281208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.281697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.283216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.284960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.287204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 55/108] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:57.288180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.301575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.307076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:57.308308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:57.380431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.386625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 54/109] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:57.393053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cc409-682b-4f50-9e56-23d54ce88b18 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.394599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.428547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.429695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.430243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.430714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.432150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.433953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.435949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 56/110] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:57.436933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.518566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fc4b2-d9ec-405b-8497-a1e7c6c09547 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.523370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.544834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ead57-5113-4682-8202-9034b3bb7562 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.555496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.556328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.556650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.556953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/15a2f040e2f14226954b7f7b777d1ead/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.559926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, user_id=15a2f040e2f14226954b7f7b777d1ead, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.562002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.582913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.585921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.585921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.585921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.595445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:57.603534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:57.604474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:57.605331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 57/111] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:57.606084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.643220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 55/112] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:03:57 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/15a2f040e2f14226954b7f7b777d1ead/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:03:57.644814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.729825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120223-4f1f-408d-a13b-f74daaccd24a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.734485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.769669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.771256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.771256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.771479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.779471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:57.784417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:57.787826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6bce0-a28d-4ab7-8b25-c1f5645e0f56 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:57.793140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:57.794968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:57.795720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 58/113] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:57.796349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.854787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.855757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.856044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.856586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.857306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.859247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdcaf4-8008-424d-8d2b-66777c4ca229 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.914077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bd6c2-7f36-417d-852e-44b34533ef34 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:57.950620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:57.951514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.951903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.952197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.953192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:57.953617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:57.955562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c1a37-2649-4132-a7aa-495b9b1ea2fb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:57.957249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 56/114] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:03:57 2022] POST /identity/v3/projects => generated 302 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:03:57.958796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa685c-d5fd-447d-a0fe-9c971bd1770f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.959277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa685c-d5fd-447d-a0fe-9c971bd1770f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.959727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa685c-d5fd-447d-a0fe-9c971bd1770f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:57.961204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 57/115] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:57.962830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa685c-d5fd-447d-a0fe-9c971bd1770f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:57.974139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 59/116] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:57.978423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9005-e7fa-4a92-8f8d-36dabf3918c7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:57.978423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9005-e7fa-4a92-8f8d-36dabf3918c7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:57.978658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9005-e7fa-4a92-8f8d-36dabf3918c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.019304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f9cd9005-e7fa-4a92-8f8d-36dabf3918c7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:58.020557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f9cd9005-e7fa-4a92-8f8d-36dabf3918c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:58.110123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa685c-d5fd-447d-a0fe-9c971bd1770f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.120207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.151240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.153031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.153502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.154502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.155436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:58.156393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:58.159593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:58.172043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.172353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 58/117] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:03:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:58.178683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.184686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 60/118] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:03:58.186240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620074-8a7c-4559-9c21-8208343dd183 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.186531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620074-8a7c-4559-9c21-8208343dd183 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.186807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620074-8a7c-4559-9c21-8208343dd183 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.188436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 61/119] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:03:58.188991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620074-8a7c-4559-9c21-8208343dd183 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.293240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e016-086c-4d30-bff6-70dec02688d1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.318110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620074-8a7c-4559-9c21-8208343dd183 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.332298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.333631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.333929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.334238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.335698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:58.337780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c79c-51ca-4396-8e67-9d1340f8c351 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:58.339521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 59/120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:58.342185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.342690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.343153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.364680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.365622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.365965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.366289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] PATH_INFO: `/v3/users/75b194f161ba4447afdfde4c23b696e6` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.366950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:03:58.368284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:03:58.384551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] RBAC: Authorizing `identity:get_user(user_id=75b194f161ba4447afdfde4c23b696e6)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:58.386063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:58.387948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 62/121] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/users/75b194f161ba4447afdfde4c23b696e6 => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:58.388545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.414001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.445698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.459449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 60/122] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:03:58.466285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.526780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2593be-c902-4493-ad7f-d5c7e9bc5f14 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.575703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.576991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.577665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.578189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.586405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cbfc6-0980-4e17-8b77-ec63378b99b1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.592478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:58.604015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:58.605052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:03:58.606008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:03:58.608446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 63/123] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:58.609409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.628149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.629029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.629459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.629673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.630981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:58.632421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:58.634399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 61/124] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:58.635153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.681649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.732532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b266e-8ce4-4231-b12b-707ebe7fb1bc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.742398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.759132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134400-ac7a-42fd-b635-a897f5af5040 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.769835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.770645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.771355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.771469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.802251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:58.803533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:58.804396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:58.804831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:58.819598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:03:58.829942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:03:58.830983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:03:58.831931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:03:58.834423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 62/125] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:03:58.839390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.856376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:58.857893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:58.962635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:58.968866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188720-1218-4ad4-8041-36a778311d2e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:58.974765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 64/126] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:03:58.975711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:58.994355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.018180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.020438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.020438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.020438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.020893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:59.021431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.023830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.034703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 63/127] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:03:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.036282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.085757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7679c9-788f-4f40-b466-a93110f84238 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.123296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.124327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.124729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.125143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/a03bfabd375944c7923bb6e092aa3498/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.127500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, user_id=a03bfabd375944c7923bb6e092aa3498, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.129070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.149335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f6a10-859e-433c-a176-95ed0638779d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.185510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.186281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.186610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.186918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.187698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:59.188028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.189407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.202132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.202132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.225851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 65/129] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:03:58 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/a03bfabd375944c7923bb6e092aa3498/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:03:59.226986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.253781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.369342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.401592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0638-c949-4dde-97fb-fc65d48a4630 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.435551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a592f-2ecc-4548-9250-e06ca2306cbb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.448735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.448735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.448735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.448735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.449078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.449078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.449078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 66/130] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.449506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.476028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.476939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.477225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.477512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.479038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.480637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.482665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 65/131] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.483285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.580584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17c862-2778-4089-98c4-ffd06ba4fea0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.584620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.595125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b0156-7681-416a-9d72-1818ec9511ad None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.628905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.630352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.630756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.631044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.632566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.634421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.634799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.635845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.636194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 67/132] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.636437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.636874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.636987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.638329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.639930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.642340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 66/133] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.644982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.645747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.759145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34b3d5-ff0b-420a-8854-492e5ea4596f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.799199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.800123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.800416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.800699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.801549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=75b194f161ba4447afdfde4c23b696e6) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:03:59.801867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.804293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.810897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 68/134] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/role_assignments?user.id=75b194f161ba4447afdfde4c23b696e6&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.811703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:03:59.892353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6a7d3-61e2-440d-86ec-4238f65695d5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.895782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:03:59.941183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb465f1-4aa7-4b0c-aee6-46028a16feb5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:03:59.971686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.972971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.973463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.973944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] PATH_INFO: `/v3/projects/e40e053c1881470fa8cbf538026212e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.979025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] RBAC: Authorizing `identity:get_project(project_id=e40e053c1881470fa8cbf538026212e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.980742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.983037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:03:59.983305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 67/135] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:03:59 2022] GET /identity/v3/projects/e40e053c1881470fa8cbf538026212e2 => generated 312 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:03:59.986047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:03:59.986047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:03:59.986047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/15a2f040e2f14226954b7f7b777d1ead/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:03:59.991201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, user_id=15a2f040e2f14226954b7f7b777d1ead, role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:03:59.992810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b974e3e-938f-4d90-acfa-8fefafeb814d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:03:59.999792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:00.030912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:00.086055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 69/136] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:03:59 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/15a2f040e2f14226954b7f7b777d1ead/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:00.118994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adff1a-28ad-4d52-bb62-bc30009c2653 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:00.144076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c86-9132-48af-bcf3-51ab663b95fa None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.144393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c86-9132-48af-bcf3-51ab663b95fa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.144652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c86-9132-48af-bcf3-51ab663b95fa None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.146321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 70/137] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:00.152069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.152357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.152670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.160108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:00.161077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.161363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.161641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] PATH_INFO: `/v3/projects/e40e053c1881470fa8cbf538026212e2/users/15a2f040e2f14226954b7f7b777d1ead/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.163897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e40e053c1881470fa8cbf538026212e2, user_id=15a2f040e2f14226954b7f7b777d1ead, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:00.165646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:00.171498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:00.172762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:00.239075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 68/138] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:03:59 2022] PUT /identity/v3/projects/e40e053c1881470fa8cbf538026212e2/users/15a2f040e2f14226954b7f7b777d1ead/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:00.256681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 71/139] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:00.280459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:00.403658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:00.425545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833fac6-5e9c-49ee-bc91-ab7277cab9e0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:00.475127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:00.476289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.476708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.477129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.492329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:00.505839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:00.506818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:00.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:00.508201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 69/140] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:00.514420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:00.594693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:00.644792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7093-bb50-4647-85e7-97dc1969ed5a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:00.692675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:00.693295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.693578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.693859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.694647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:00.694962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:00.696360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51df1f-7f93-4b01-bae2-2c1a5aba4f26 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:00.705927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 72/141] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:00.711740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:00.836545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395505-e208-4d72-95b9-b9bd4f83e342 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.836728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395505-e208-4d72-95b9-b9bd4f83e342 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.836952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395505-e208-4d72-95b9-b9bd4f83e342 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.838383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 73/142] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:00.861395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.861395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.861395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.868712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f18fa-4d88-4176-9720-b27186c5c7af None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:00.902958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:00.904311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:00.923630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:00.924888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:00.925693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:00.926203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:00.930952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:00.932811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:00.934900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 70/143] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:00 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:00.941045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:00.970603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:01.004021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 74/144] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:01.028996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:01.082723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:01.098663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0cf6-1544-4e70-9c7e-a482b00ee5b0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:01.134839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98986-0559-4a7a-8477-e63cbd054106 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:01.159321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:01.159991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:01.160283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:01.160565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:01.171979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:01.173026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:01.173449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:01.173859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:01.175234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:01.176954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:01.178781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 75/145] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:01.184571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:01.298071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ef7-5949-4d14-9df2-9750ea8952e8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:01.317107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:01.335642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:01.336551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:01.336973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:01.337384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:01.338409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:01.339999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:01.359052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:01.360553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:01.383245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:01.447551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 76/146] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:01 2022] POST /identity/v3/users => generated 283 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:04:01.493805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 71/147] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:01.499515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:01.669755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7647-4590-4d84-b518-9a74fccbb43b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:01.718472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:01.719384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:01.719664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:01.719937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:01.721130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:01.722620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8109d-8b6a-42cd-af2d-85992b1383ef None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:01.724514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 77/148] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:01 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:01.730512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:01.864297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cc1e2-ce84-49de-a9f2-b4de68008b63 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:01.889278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:01.914681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:01.915358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:01.915657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:01.915936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:01.916789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:01.917093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:01.918562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c78d5-307f-4771-bc98-146517f81c1b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:01.930772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 72/149] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:01 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:02.491271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114020-47c0-4fb8-a2d8-0cd7326b40e3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:02.491603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114020-47c0-4fb8-a2d8-0cd7326b40e3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:02.491781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114020-47c0-4fb8-a2d8-0cd7326b40e3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:02.493283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 78/150] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:02.498941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:02.499423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:02.499498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:02.520561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:02.521755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:02.606993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 73/151] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:02.643223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93114020-47c0-4fb8-a2d8-0cd7326b40e3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:02.696536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:02.756691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93114020-47c0-4fb8-a2d8-0cd7326b40e3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:02.791386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:02.792238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:02.792522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:02.792796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:02.805978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:02.814739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:02.815661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:02.816435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 79/152] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:02.823680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:02.935126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dafca-8915-4741-a860-423ddb972fa9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:02.946135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:02.979813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:02.980492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:02.980771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:02.981121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:02.981857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:02.982644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:02.984043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9ffe-2ebc-4dfe-a4b6-e186056b912d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:02.996523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 74/153] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:03.002868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:03.095758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421a04-5688-4f48-ab40-6eb4053da381 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.096050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421a04-5688-4f48-ab40-6eb4053da381 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.096228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421a04-5688-4f48-ab40-6eb4053da381 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.098009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 75/154] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:03.107614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.107903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.108180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.127382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69c82-ab19-460d-81b9-8168597486ac None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:03.129315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:03.134857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:03.136041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:03.165120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:03.166557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.168342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.168787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] PATH_INFO: `/v3/users/a03bfabd375944c7923bb6e092aa3498` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.170835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] RBAC: Authorizing `identity:get_user(user_id=a03bfabd375944c7923bb6e092aa3498)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:03.172719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:03.174997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 80/155] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:03 2022] GET /identity/v3/users/a03bfabd375944c7923bb6e092aa3498 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:03.183237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:03.298441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 76/156] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:03.350442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:03.357829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0fc9a-b55c-497f-b7c4-450215f1ac4c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:03.406455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:03.407389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.407814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.408229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.476490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:03.529429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:03.531349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3ba1f-44b9-490f-abff-7682a4fd17d0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:03.583099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:03.591928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334beb6c-d3dd-4aca-8a3f-37c99efaa81c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:03.645267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:03.646385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.646807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.647221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.658271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a4eb7fb7-ebee-406f-a574-091a468632df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:03.672530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:03.673596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:03.674579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:03.681080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 77/157] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:03.682596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbddaae-df9f-4390-a03b-2dc317ed3ee0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.683027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbddaae-df9f-4390-a03b-2dc317ed3ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.683444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbddaae-df9f-4390-a03b-2dc317ed3ee0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.684887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 78/158] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:03.685746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbddaae-df9f-4390-a03b-2dc317ed3ee0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:03.804687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 81/159] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:03.806723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.807192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.807644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.845644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbddaae-df9f-4390-a03b-2dc317ed3ee0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:03.879059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:03.880629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:03.904178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:03.904787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:03.905069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:03.905346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:03.906648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:03.906956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:03.908329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:03.921203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 79/160] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:03.923887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:03.960232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.007345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.012635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 82/161] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:04.013824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.093526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e56ea-ceee-4132-a9b3-89872933d953 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.159649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.161039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.161469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.162074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.163755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:04.165517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8b85d-ee6d-42fa-904c-ad8cd89a2538 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:04.167392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 80/162] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:03 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:04.168766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910560-4ec6-4464-8a15-b1811ae2616f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.169196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910560-4ec6-4464-8a15-b1811ae2616f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.169613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0910560-4ec6-4464-8a15-b1811ae2616f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.171413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 81/163] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:04.172427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0910560-4ec6-4464-8a15-b1811ae2616f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.198702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779590b-6bb8-4e1c-85f1-8bd17fdc248b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.266409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.268224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.268509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.268784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] PATH_INFO: `/v3/users/75b194f161ba4447afdfde4c23b696e6` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.272113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] RBAC: Authorizing `identity:get_user(user_id=75b194f161ba4447afdfde4c23b696e6)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:04.274357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:04.276188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 83/164] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/users/75b194f161ba4447afdfde4c23b696e6 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:04.279901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.356884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.370263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0910560-4ec6-4464-8a15-b1811ae2616f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.397447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.425908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.427885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.427885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.427885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.446054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-43dd0be3-e1dd-4fec-a86e-f246d4a607ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:04.451856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:04.453643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:04.454636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:04.455846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b645-c03d-4929-896c-17be926a5266 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.456686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 82/165] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:04.462037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.462037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.462037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.494279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:04.495478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:04.501497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=a03bfabd375944c7923bb6e092aa3498) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:04.508579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:04.517885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 84/166] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/role_assignments?user.id=a03bfabd375944c7923bb6e092aa3498&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:04.521814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.646670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 83/167] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:04.647636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.734140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28696039-3503-453f-ae9b-f3756952fc6b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.770408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.790579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4a9c-820d-488d-b4fb-9d54e0ca91e7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.816593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:04.827023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.827705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.828017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.828294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.829053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:04.829358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:04.830936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:04.834747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:04.835638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:04.835922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:04.836195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:04.837516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:04.839033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:04.839939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 84/168] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:04.840851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 85/169] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:04.840959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.847006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:04.966284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f643-2805-4d1a-a873-ee5da51499d0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:04.983744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d0ed2-a507-4f06-bdd2-f1933c4d9a94 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.010721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.011471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.011752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.012023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.026045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:05.033939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:05.034849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 85/170] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:05.036042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.039376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.040560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.041679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.042190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.042654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.047307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.049019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.051163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 86/171] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:04 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:05.054916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.145503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b60c0-a05a-448a-8236-af5cb7e7d340 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.171163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.183204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.186037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.186037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.186037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/75b194f161ba4447afdfde4c23b696e6/roles/637edfd3fa00487b93d8c9cb36e578d8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.186683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, user_id=75b194f161ba4447afdfde4c23b696e6, role_id=637edfd3fa00487b93d8c9cb36e578d8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.188184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.225863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838254b-8220-4619-bbb3-76ad38cfb38b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.262551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 86/172] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:05 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/users/75b194f161ba4447afdfde4c23b696e6/roles/637edfd3fa00487b93d8c9cb36e578d8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:05.263786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.283784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.284826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.285366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.285819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.286867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:05.287373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.290050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.308404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 87/173] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:05.313788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.390424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.465040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb920fa2-f8e8-4457-80a4-3515dbf8ae01 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.530614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476cfe2-d5e5-4b0b-a6ca-fcc29c8e31ba None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.542252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.542917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.543198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.543473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.587326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.598720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.599607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.599890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.600163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] PATH_INFO: `/v3/users/15a2f040e2f14226954b7f7b777d1ead` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.602704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=15a2f040e2f14226954b7f7b777d1ead)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.604588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.606675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 88/174] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:05 2022] GET /identity/v3/users/15a2f040e2f14226954b7f7b777d1ead => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:05.615218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.670526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.672119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.767156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.815421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 87/175] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:05.820903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:05.831670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ebc1-80b4-472b-b688-6d63acbfe0d6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.871432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.872518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.872635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.872896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.941627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7075-0ed3-4d9c-9690-771e808ff6db None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:05.942152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:05.946048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.947545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a215-2437-443d-af28-44e160d0c2c4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.977794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:05.978943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.979225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.979495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] PATH_INFO: `/v3/projects/e40e053c1881470fa8cbf538026212e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.980753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] RBAC: Authorizing `identity:get_project(project_id=e40e053c1881470fa8cbf538026212e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:05.982233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8683ee-d662-4ee2-8fa7-7da613a24316 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:05.984159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 88/176] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:05 2022] GET /identity/v3/projects/e40e053c1881470fa8cbf538026212e2 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:05.989837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96264-5932-42ee-a84a-78b973a97f46 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.990161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96264-5932-42ee-a84a-78b973a97f46 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.990447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96264-5932-42ee-a84a-78b973a97f46 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.991829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 89/177] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:05.992767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122bfcf-684f-4078-80e6-863b23c80731 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:05.993051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122bfcf-684f-4078-80e6-863b23c80731 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:05.993328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122bfcf-684f-4078-80e6-863b23c80731 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:05.998600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 90/178] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:05.999598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea875-b339-4da6-9a33-ee8a0d9422ac None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.000408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea875-b339-4da6-9a33-ee8a0d9422ac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.000690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea875-b339-4da6-9a33-ee8a0d9422ac None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.005962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 91/179] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:06.005962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea875-b339-4da6-9a33-ee8a0d9422ac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.037632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.058592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 89/180] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:06.059825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326c373-efdb-4061-89ba-aa671dfdee62 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.060117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326c373-efdb-4061-89ba-aa671dfdee62 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.060389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326c373-efdb-4061-89ba-aa671dfdee62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.077080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c326c373-efdb-4061-89ba-aa671dfdee62 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:06.078894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c326c373-efdb-4061-89ba-aa671dfdee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:06.117115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea875-b339-4da6-9a33-ee8a0d9422ac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.145348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 90/181] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:06.146843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.147133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.147405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.162310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.163402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:06.163515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.164097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.164574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:06.164696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.165779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=e40e053c1881470fa8cbf538026212e2, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:06.166120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:06.168673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788fbd9-6414-4a7f-8329-5785816edefc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:06.175909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 92/182] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=e40e053c1881470fa8cbf538026212e2&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:06.177632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.177924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.178227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.189412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.201868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:06.204907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:06.242410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 91/183] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:06.243504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.336388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 93/184] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:06.337216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.368606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.401201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44e38f-517f-4496-a3c3-fe4c15a1dce0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.446665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.447622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.447913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.448191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.450853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:06.452711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:06.455986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 92/185] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:06.455986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.470912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70c7a5-9e7e-478b-8ca1-24accef8dd0e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.487320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.517208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.518057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.518316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.518588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.528959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:06.536745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:06.537628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 94/186] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:06.539061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.572359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f1c5e-58f0-4f77-939a-bb0e68cb5249 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.607906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.608647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.608932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.609761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.620699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:06.628851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:06.629714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:06.630607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 93/187] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:06.633557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.634428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.650479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201f08-2a51-457f-a62e-812e5049aa50 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.689744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.690568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.690853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.691124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.692296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:06.693744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:06.695534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 95/188] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:06.696131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.729118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.744257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ba4d7-6b30-4cac-a540-9ad90386ea05 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.779838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.780500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.780784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.781586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.782459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=15a2f040e2f14226954b7f7b777d1ead) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:06.782769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:06.784207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:06.789948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 94/189] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/role_assignments?user.id=15a2f040e2f14226954b7f7b777d1ead&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:06.789948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.815752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b9bd4-156d-42d2-8953-60b4383a39d9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:06.866829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:06.867519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:06.867797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:06.868073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:06.868888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:06.869196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:06.870721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:06.884698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 96/190] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:06.884698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:06.901209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:06.961506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b33c52-b099-48eb-ab34-4db854f89d1c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.001427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.002210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.002390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.002658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.003491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:07.003746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.005073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.005637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa7fda-fa77-41c0-bfe5-c29556168c60 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.014046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:07.015026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.031776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.044762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.045446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.045736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.050526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.050526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.050526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.130649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acf274-361f-4ccc-9b1c-56753dd132f3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.166052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 97/192] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:06 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:07.166052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.174078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.175083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.175366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.175637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.194229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.195740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.197824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 96/193] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:07.204109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.259821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.287738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.297795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96424-8770-44b2-98d1-971d9d14bc6e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.321727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1665a2-82d5-4b98-86c6-551638865f67 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.336101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.337065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.337351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.337627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.339347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.342367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.342995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 98/194] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:07.349322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.367527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.368670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.369695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.369695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.452829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.454856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.475839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5181b-ffd8-47c6-92ae-c619cfbefc18 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.519243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.520041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.520445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.520673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.529910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.534958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.567279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 97/195] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:07.573394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.610164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.611934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.704255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380192b-91b2-4f84-ac9e-ba3c60641b1e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.739195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 99/196] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:07.741710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.746499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.746499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.746499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.746739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.750321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.750321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.752504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 98/197] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:07.762864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.810533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.855838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:07.870584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8991216-1b2d-4489-a332-124ccbf5f014 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:07.945653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:07.946855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:07.948903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:07.949355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:07.951873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:07.955699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:07.957730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 100/198] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:07.963504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:07.974378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c55b-2733-4556-9e37-9e9becd65d4b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:08.025607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:08.026396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.026749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.027096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.028002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=7a8999a909ac4e20978e812b4608926f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:08.028378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:08.029902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e367-21ab-4026-8f21-85c9fa8cd7bf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:08.039956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 99/199] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/role_assignments?user.id=7a8999a909ac4e20978e812b4608926f&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:08.039956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f450e-7d36-4185-8174-a00c6792960e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.039956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f450e-7d36-4185-8174-a00c6792960e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.039956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f450e-7d36-4185-8174-a00c6792960e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.039956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 100/200] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:08.050040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.050040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.050040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.066110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:08.066635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:08.164519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d3cf4-c4da-4e40-9c75-784c642b7595 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:08.179006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:08.226985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 101/201] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:08.238774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:08.239457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.239742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.240084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.240936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=7a8999a909ac4e20978e812b4608926f) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:08.241249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:08.242747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:08.258678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:08.262110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 101/202] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:07 2022] GET /identity/v3/role_assignments?user.id=7a8999a909ac4e20978e812b4608926f&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:08.308317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:08.398533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0440878-87de-4c9e-8bb6-6c665d3d8cc0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:08.448709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:08.449539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.449797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.450098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.463815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:08.471953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:08.472794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:08.473614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:08.475952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 102/203] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:08.483009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:08.588559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acddf1-1bdd-4cc7-9b86-c57c7486bf3c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:08.629421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:08.630476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.630956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.631421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.632400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:08.632912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:08.634498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:08.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 102/204] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:08.648814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:08.769443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:08.782932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7bb01-37ca-4830-a451-bcefa94cf20c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:08.844142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:08.845031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:08.845313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:08.845585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] PATH_INFO: `/v3/users/75b194f161ba4447afdfde4c23b696e6` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:08.846851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=75b194f161ba4447afdfde4c23b696e6)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:08.848319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:08.849964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 103/205] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:08 2022] GET /identity/v3/users/75b194f161ba4447afdfde4c23b696e6 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:08.856072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:08.919072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:09.079203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b900-e0b4-47f8-8f04-5813841913ae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:09.147782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:09.148444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:09.148724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:09.148995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:09.241059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:09.242665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:09.285975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:09.370380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 103/206] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:09.379111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:09.530096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6546fa1-c3a9-42fe-b798-7e0edd5c6fc0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:09.576985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:09.578406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:09.578690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:09.578965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:09.580221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:09.581793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b687-54c7-4440-ae5f-53b21983d03a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:09.583859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 104/207] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:09 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:09.589424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:09.705203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03b63-c781-4000-a6e0-85dfe131ab0e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:09.744744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:09.750287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:09.750953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:09.751236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:09.751509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:09.752341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, scope.project.id=aca8271cb9e64af39cb1d04d5246214d, user.id=75b194f161ba4447afdfde4c23b696e6) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:09.752653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:09.754130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a137f2-82e8-497a-9f87-77217d06fd11 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:09.762216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 104/208] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:09 2022] GET /identity/v3/role_assignments?user.id=75b194f161ba4447afdfde4c23b696e6&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:10.124211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791358-0e9a-4550-a4b2-0a999b7657da None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.124560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791358-0e9a-4550-a4b2-0a999b7657da None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.125339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791358-0e9a-4550-a4b2-0a999b7657da None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.126771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 105/209] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:10.132020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.132848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.133335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.148413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290acb73-dd3d-4d4e-a642-415c901f0eb7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.149005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290acb73-dd3d-4d4e-a642-415c901f0eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.149477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290acb73-dd3d-4d4e-a642-415c901f0eb7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.150491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:10.151329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 106/210] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:10.154057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:10.156964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.157246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.157523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.176207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:10.177766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:10.207742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.240932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 105/211] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:10.265158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.267211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 107/212] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:10.292699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.370279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c4d5e-6154-4b1e-ad0a-81e4855b5b45 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.397092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.398563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93e45-19a5-4022-bbdd-7b77ae24183a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.407268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.410859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.410859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.410859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.419828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:10.429420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:10.430363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:10.431136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:10.433483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 106/213] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:10.439256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.441748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.442643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.442928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.443252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.444422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:10.446148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:10.447970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 108/214] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:10.453485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.483203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.550646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd62e06-77c1-42c4-b179-c56810419d0d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.565073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6abbf6-dd9a-49f8-bb32-36e10cb012a5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.585563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.586543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.586981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.587397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.588326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:10.588811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:10.590399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:10.599731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 107/215] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:10.602900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.604071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.604627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.605135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.605461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.606386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:10.608035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fddea-74ff-43ed-948f-44b7b1a9dbd7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:10.656637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.706293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe79db4-b693-4259-813f-eefd0b2192ae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.720394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 109/216] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:10 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:10.732057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a23d6-6cc7-4af0-a994-4b68d505bbe1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.732351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a23d6-6cc7-4af0-a994-4b68d505bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.732633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a23d6-6cc7-4af0-a994-4b68d505bbe1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.734632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 110/217] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:10.742102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.742102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.742102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.747041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.748204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.748494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.748823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] PATH_INFO: `/v3/users/caa2ee2b749042e8b96ebfd4911b111c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.763018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:10.764621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:10.772182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=caa2ee2b749042e8b96ebfd4911b111c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:10.773678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:10.776490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 108/218] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/users/caa2ee2b749042e8b96ebfd4911b111c => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:10.782923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.854163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 111/219] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:10.870634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:10.886861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.902270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ba0c0-a9bf-4a35-a89f-02b0d9202cd1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:10.911445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:10.941255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:10.942188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:10.942470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:10.942743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:10.991955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f2e57-c814-4990-b671-187e87fcf5c7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.036180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.036981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.037256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.037526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.037633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.039077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.052993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0b60e321-358b-4f2a-99ac-f494d5650134 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:11.061690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:11.062752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:11.063542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 112/220] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:11.064909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bf2cb-682a-4715-849b-c3d7097c5ed9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.064909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bf2cb-682a-4715-849b-c3d7097c5ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.064909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bf2cb-682a-4715-849b-c3d7097c5ed9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.066942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 113/221] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:11.072515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.072800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.073073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.093233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:11.095360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:11.162365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 109/222] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:11.163829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.173108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.182756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 114/223] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:11.184016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.199621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.292304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b117c-ce95-4caf-96b4-3a6a1e287152 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.328364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.329484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.330003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.330501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.331447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:11.331952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.333550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.334573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35435cde-642e-49e2-9359-6b06b779bb43 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.342941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 110/224] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.344041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.379184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.380062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.380345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.380620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.382989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.384661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.388419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 115/225] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.389455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.418951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.462572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14dee2-09f0-4206-84f6-9f09a1baada1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.490995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.505862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.507067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.507856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.508283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.521728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:11.527840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce8f1d-26c4-4594-ad59-c4c93bd1e5cf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:11.532314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:11.533400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:11.534246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:11.536292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 111/226] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:11.537681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.574878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.576304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.576709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.577290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.578905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.581138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.584410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 116/227] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.585277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.662573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b923-7d95-417f-af60-c298f93d5740 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.692585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.697571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4cef-a497-4862-b4bd-ea38d3472daa None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.701479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.701943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.702297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.702576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.703454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=caa2ee2b749042e8b96ebfd4911b111c) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:11.703771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.705237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.711398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 112/228] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/role_assignments?user.id=caa2ee2b749042e8b96ebfd4911b111c&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.712155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.738512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.739649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.740198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.740668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.741608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:11.750048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.753199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.767605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:11.772734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 117/229] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.779125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.888784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afa59f-3770-450f-91a7-1d07d1c5aa31 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:11.932322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:11.933266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:11.933544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:11.933812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:11.938754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:11.940211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:11.942109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 113/230] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:11.947807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:11.979642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc967d-3b97-48fc-83a5-3e366cf3286a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:12.013776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:12.035725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:12.036683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:12.036961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:12.037302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:12.038641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:12.040004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:12.041929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 118/231] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:11 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:12.047303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:12.064475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1ac2-34bd-4740-b637-49274aa9fd51 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:12.115367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:12.128480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:12.129559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:12.131905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:12.131905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/7a8999a909ac4e20978e812b4608926f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:12.132769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, user_id=7a8999a909ac4e20978e812b4608926f, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:12.134427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c3ecf-6a52-46d7-b38e-74da83282949 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:12.155129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712e82-3882-4bae-8ec1-e1cf5e607d6f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:12.194672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:12.195533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:12.195955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:12.196233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:12.197464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:12.199027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:12.201106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 119/232] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:12 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:12.207057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:12.233239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 114/233] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:11 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/7a8999a909ac4e20978e812b4608926f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:12.333405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf297c-b677-4a02-8d19-b59cea191850 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:12.371429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:12.372327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:12.372604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:12.372876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/7a8999a909ac4e20978e812b4608926f/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:12.376041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, user_id=7a8999a909ac4e20978e812b4608926f, role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:12.377563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c36aa-78ad-4b63-b100-130b6a464d0d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:12.381237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:12.506568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 120/234] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:12 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/7a8999a909ac4e20978e812b4608926f/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:04:13.295357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff04-c2cc-4bb2-9955-a9eb5ea21bff None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.296320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff04-c2cc-4bb2-9955-a9eb5ea21bff None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.297323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789ff04-c2cc-4bb2-9955-a9eb5ea21bff None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.298923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 115/235] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:13.304616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.305166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.305661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.335060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:13.336616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:13.380558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734cb8a-8d3a-4ba2-a0b7-6a2253ef84ab None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.380885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734cb8a-8d3a-4ba2-a0b7-6a2253ef84ab None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.381119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734cb8a-8d3a-4ba2-a0b7-6a2253ef84ab None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.382681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 116/236] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:13.387502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.387791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.388068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.418224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:13.419417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:13.425228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 121/237] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:13.462041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:13.529520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 117/238] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:13.568691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:13.575087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a152-b7c4-46b4-8b95-779cf619a7d8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:13.606368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:13.621004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:13.621812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.622113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.622381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.625674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:13.627286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:13.631984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 122/239] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:13.639543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:13.674382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:13.729962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875797f9-428c-442f-b7f8-d000d8e14587 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:13.768585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:13.769561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.769909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.770326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.782236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:13.791180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:13.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:13.792837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 118/240] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:13.799035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:13.909634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9f6f0-224b-4281-8f2f-7b52507e9aa9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:13.922333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:13.946648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e72f-0bc0-4efb-952f-0e9c28f5f1a1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:13.946969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:13.947671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.947955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.948389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.949141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:13.949453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:13.950921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:13.962423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 119/241] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:13.967043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:13.991194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:13.991957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:13.992352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:13.992704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:13.993538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:13.995096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.074558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a560274-f2e1-4677-a7a0-9b25e3c64da5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.088867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 123/242] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:13 2022] POST /identity/v3/users => generated 293 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:14.111099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.112004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.112283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.112553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] PATH_INFO: `/v3/users/f45a681ed7c4464d97e7de8c31e19928` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.128377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] RBAC: Authorizing `identity:get_user(user_id=f45a681ed7c4464d97e7de8c31e19928)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:14.129853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57fbd3-a2c8-422d-afb1-7d7616b5b955 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.131978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 120/243] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:13 2022] GET /identity/v3/users/f45a681ed7c4464d97e7de8c31e19928 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:14.137394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:14.227695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:14.301998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723147d-f28f-40a6-90db-59a9d7ffcd8c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.357637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.358519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.358926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.359774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.416052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d5b79-0e22-4ebb-87ab-cd08fbe5931c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.416548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d5b79-0e22-4ebb-87ab-cd08fbe5931c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.416622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d5b79-0e22-4ebb-87ab-cd08fbe5931c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.418106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 121/244] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:14.423464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.423747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.424028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.435834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:14.437561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.441235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:14.442436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:14.466318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:14.497430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:14.517048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 122/245] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:14.544288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:14.559417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 124/246] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:14.565291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:14.673708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c014b8f-5e4a-4f0b-b865-af44c6b45ad3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.687976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677d822-1594-4365-8d19-fe9c987f2b7a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.713996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.715048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.715424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.715813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.730246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:14.735599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.736569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.736947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.737318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.738638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:14.740126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:14.741291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:14.742321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:14.743065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.744560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 123/247] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:14.744665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 125/248] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:14.750292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:14.752200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:14.773891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:14.785330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:14.891343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff03-79ec-45e0-9624-000d0d0a5df6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.894614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0884c-668e-48e7-8da9-23f4ef7a5911 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:14.934535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.935145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.935450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.935790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.936565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:14.936849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:14.938669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:14.939122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53c16-420d-44d6-a6a8-48ae2c31f334 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.939764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:14.940241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:14.940966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:14.942044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=f45a681ed7c4464d97e7de8c31e19928) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:14.942556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:14.944220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:14.951640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 124/249] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/role_assignments?user.id=f45a681ed7c4464d97e7de8c31e19928&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:14.957617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 126/250] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:14.963557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.051953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6d00-8e35-4052-b03b-f31617f8c096 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.052274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6d00-8e35-4052-b03b-f31617f8c096 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.052506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6d00-8e35-4052-b03b-f31617f8c096 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.054173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 127/251] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:15.082878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:15.085227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.085725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.086220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.095789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc50ee-1e87-491a-88e9-7991a96a1efd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.119186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:15.120770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:15.141475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.141475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.141475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.141475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] PATH_INFO: `/v3/users/caa2ee2b749042e8b96ebfd4911b111c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.141931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] RBAC: Authorizing `identity:get_user(user_id=caa2ee2b749042e8b96ebfd4911b111c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:15.141931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:15.141931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 125/252] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:14 2022] GET /identity/v3/users/caa2ee2b749042e8b96ebfd4911b111c => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:15.147748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.254439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925a72-afd0-44a2-b03f-7d54af35208e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.264880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 128/253] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:15.302813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.303711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.303989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.304272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.305546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:15.307044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:15.308927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 126/254] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:15.315265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.327080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.340468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:15.366424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:15.442630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3596d2-3662-4e1b-9a6c-bf08f784d3e1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.478522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.479253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.479534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.479807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.490785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-350ded9e-8013-4918-a074-71a3e6cf7d72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:15.498456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:15.499424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:15.500720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:15.504956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 127/255] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:15.506385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338342f-6a35-4cbc-976b-7e055718c467 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.506675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338342f-6a35-4cbc-976b-7e055718c467 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.506952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338342f-6a35-4cbc-976b-7e055718c467 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.508289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 128/256] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:15.509375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338342f-6a35-4cbc-976b-7e055718c467 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.540962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2013e1-628e-409b-bbad-2c99d857368f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.608033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.609484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.609767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.610051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/caa2ee2b749042e8b96ebfd4911b111c/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.613364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338342f-6a35-4cbc-976b-7e055718c467 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.615338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, user_id=caa2ee2b749042e8b96ebfd4911b111c, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:15.616803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:15.634436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:15.648203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.649027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.649429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.649806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.650741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:15.651146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:15.652659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34692b13-a2d4-4295-8c26-799442f2aa15 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:15.666430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 129/257] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:15.668135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.668543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.668913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.686816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:15.688108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:15.699193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 129/258] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:15 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/caa2ee2b749042e8b96ebfd4911b111c/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:15.700033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.832387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 130/259] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:15.851634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:15.868405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:15.878063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfd369-111d-4cbd-a510-ddc995ad6695 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:15.916856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:15.917744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:15.918040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:15.918324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:15.919598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:15.921133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:15.923543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 130/260] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:15.929151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:16.016851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:16.060068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bbe33-cfef-49b1-98e9-73558d308ca1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.060494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288b09-550a-4326-965d-89b521aac5b4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.098859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.099639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.099997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.100374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.103270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.104703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.105185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.105653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.119747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:16.126973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:16.137511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:16.138519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:16.139303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:16.146052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 131/261] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:16.151574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:16.182554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:16.184284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:16.334806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 131/262] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:16.340730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:16.393223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:16.464000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cea8f-a06b-404f-88e9-91dfc4a99408 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.493036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f16fad-114f-43fe-8a6f-d7cd97e02d2e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.521653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:16.546456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.547210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.547500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.547777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.548535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:16.548841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:16.550251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:16.553063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.553940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.554260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.554534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.558747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:16.560388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:16.562818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 132/263] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:16 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:16.566980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 132/264] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:16.568392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:16.578639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:16.740336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d77-3372-4d49-9e75-71e584477dfe None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.753669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d339d3-cdc5-48e6-9123-4149001380d9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:16.761236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:16.806740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.807415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.807696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.807967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.808807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=7a8999a909ac4e20978e812b4608926f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:16.809117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:16.810610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:16.817442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 133/265] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:16 2022] GET /identity/v3/role_assignments?user.id=7a8999a909ac4e20978e812b4608926f&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:16.827313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:16.828254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:16.828531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:16.828802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] PATH_INFO: `/v3/users/7a8999a909ac4e20978e812b4608926f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:16.835011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] RBAC: Authorizing `identity:get_user(user_id=7a8999a909ac4e20978e812b4608926f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:16.836502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:16.838652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 133/266] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:16 2022] GET /identity/v3/users/7a8999a909ac4e20978e812b4608926f => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:16.844027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:17.012174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363634c-2ef6-4f00-9050-ffe4bed5256f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:17.074746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:17.075708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.076143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.076568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.146395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:17.191764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:17.193386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:17.394851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 134/267] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:17.403506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:17.547361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:17.590774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552ed3-fbb9-4d0c-b72e-740bf6ea6c3d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:17.635766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:17.636632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.636909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.637175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.642730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:17.644193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445ca58-b5ab-41cc-bfe6-d85b293e1854 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:17.646316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 134/268] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:17 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:17.651704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:17.767113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd362-a3be-416c-a8bc-0baea0f25f2b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.767674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd362-a3be-416c-a8bc-0baea0f25f2b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.768124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd362-a3be-416c-a8bc-0baea0f25f2b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.769690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 135/269] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:17.774866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587aba0-8616-4afb-99f8-cb19b3b90ea4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.775356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587aba0-8616-4afb-99f8-cb19b3b90ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.775806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587aba0-8616-4afb-99f8-cb19b3b90ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.776494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:17.807742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b48f3-e5da-4660-adf0-1428c5294c33 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:17.811226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9587aba0-8616-4afb-99f8-cb19b3b90ea4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:17.812700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9587aba0-8616-4afb-99f8-cb19b3b90ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:17.846124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:17.847027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.847471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.847881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.848860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=7a8999a909ac4e20978e812b4608926f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:17.849310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:17.851071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b010bb-8818-423a-b072-96315bf926b3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:17.857578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 135/270] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:17 2022] GET /identity/v3/role_assignments?user.id=7a8999a909ac4e20978e812b4608926f&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:17.951048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bae65-0897-4079-9f78-843183c0376e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.951545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bae65-0897-4079-9f78-843183c0376e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.952002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bae65-0897-4079-9f78-843183c0376e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.953367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 136/271] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 20:04:17.958330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 136/272] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:17.963731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:17.964019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:17.964298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:17.981207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:17.982450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:17.989265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bae65-0897-4079-9f78-843183c0376e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.070416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.094461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 137/273] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:18.126610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.149105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bae65-0897-4079-9f78-843183c0376e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.189410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.190501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.190930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.191340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.192646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:18.194306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:18.196545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 137/274] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:18.201804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.205365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.262102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2234f7-7c26-42e9-886b-4599bbecc671 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.309484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.311048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.311475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.312679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.326043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:18.333328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1707ad-5245-4592-973f-5f6f3d16783d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:18.336094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:18.336965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:18.337763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:18.339473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 138/275] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:18.345125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.379079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.380098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.380579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.381086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.382121 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:18.383976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6573a96-8ff5-47f0-ac5c-7bf5bf055d83 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:18.407161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.454056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f189-a1e9-45f9-8488-001624fe388b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.476495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 138/276] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:18 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:04:18.486028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666db6b-1b46-4151-892e-f7dd89892ce7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.486347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666db6b-1b46-4151-892e-f7dd89892ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.486610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666db6b-1b46-4151-892e-f7dd89892ce7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.487977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 139/277] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:18.493908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.494243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.494498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.502895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.503590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.503870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.504140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.504907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:18.505214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:18.507183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:18.519939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 139/278] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:18.521891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:18.523246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:18.525229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.591934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.613914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 140/279] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:18.634742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.641884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f79b-4cb7-4128-8326-f5e9681f6c3f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.657082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.680987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.681875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.682186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.682480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] PATH_INFO: `/v3/users/62c88931434647adb6cab231ca407e69` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.699790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] RBAC: Authorizing `identity:get_user(user_id=62c88931434647adb6cab231ca407e69)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:18.701674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:18.704124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 140/280] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/users/62c88931434647adb6cab231ca407e69 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:18.709660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.764017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de0d30-59c3-4c28-bb44-7ba2efb12d93 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.809500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.810327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.810604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.810874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.825584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:18.835330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:18.836281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:18.837068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 141/281] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:18.842778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:18.849696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d060f0-c6b7-43d5-86c0-e1d8effaf947 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:18.901863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:18.902650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:18.902929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:18.903026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:18.903202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:18.904057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, user.id=62c88931434647adb6cab231ca407e69, scope.system=all) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:18.904367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:18.906361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:18.911931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 141/282] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/role_assignments?user.id=62c88931434647adb6cab231ca407e69&scope.system=all&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:18.970108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f93fb7-41f2-4f71-8af4-5617d28cc1cb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.015393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.016104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.016385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.016662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.017432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:19.017742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.019190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd9abb-699d-43d7-9c95-e8ba53b6bc45 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.033547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 142/283] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:19.038948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.048304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64b2f9-ac34-456d-b009-34f55c8cbb31 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.048662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64b2f9-ac34-456d-b009-34f55c8cbb31 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.048860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64b2f9-ac34-456d-b009-34f55c8cbb31 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.050796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 143/284] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:19.056875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.057149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.057492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.076413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:19.078191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:19.145565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:19.161325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 144/285] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:19.185280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.220948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f47a8-5a75-47c7-8789-ac748c18790d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.232786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:19.257212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.258191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.258471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.258744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] PATH_INFO: `/v3/users/f45a681ed7c4464d97e7de8c31e19928` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.259945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] RBAC: Authorizing `identity:get_user(user_id=f45a681ed7c4464d97e7de8c31e19928)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.261379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.263916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 142/286] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/users/f45a681ed7c4464d97e7de8c31e19928 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:19.269202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.314570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608aaac-de5d-490d-8a7f-4b6826890bea None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.368668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.369521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.369800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.370084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.390700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:19.404935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:19.405899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:19.407128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 145/287] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:19.413979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.447656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851af1-c54e-46c4-a4fd-55d9de3cb752 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.453986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:19.513002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.513795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.514098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.514371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.516990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.518479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.520376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 143/288] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:19.528295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.594394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:19.597383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87880d4-f585-4b25-82bc-d201b1e1df36 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.642219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.643023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.643305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.643790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.644541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:19.644846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.646275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.655856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 146/289] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:19.661382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.709547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1a29-918a-4408-96e9-392fab08f3d2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.743715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.744602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.744881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.745153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/f45a681ed7c4464d97e7de8c31e19928/roles/637edfd3fa00487b93d8c9cb36e578d8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.747175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, user_id=f45a681ed7c4464d97e7de8c31e19928, role_id=637edfd3fa00487b93d8c9cb36e578d8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.748616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.809569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32952ce1-b362-442d-9f07-40908c144819 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:19.827604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 144/290] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:19 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/users/f45a681ed7c4464d97e7de8c31e19928/roles/637edfd3fa00487b93d8c9cb36e578d8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:19.847305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:19.848196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:19.848529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:19.848810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] PATH_INFO: `/v3/users/caa2ee2b749042e8b96ebfd4911b111c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:19.850445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=caa2ee2b749042e8b96ebfd4911b111c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:19.852000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:19.854493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 147/291] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/users/caa2ee2b749042e8b96ebfd4911b111c => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:19.860692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:19.918484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:20.007660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddac4c-5e99-446f-9392-e84df24619cf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:20.042796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:20.043902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:20.044422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:20.044889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:20.139166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:20.140698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:20.170049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:20.241902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 145/292] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:20.247489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:20.281290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:20.355185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dc919-fa31-4c0b-a073-7ee6c0735cf9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:20.391499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:20.392282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:20.392561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:20.392835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:20.394043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:20.395499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5ca17-826e-4db5-a222-d2434c89ba82 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:20.397539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 148/293] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:20 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:20.403194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:20.535874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb42ef-0973-4d25-9835-9d84da063b13 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:20.567042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:20.576467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:20.577886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:20.578379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:20.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:20.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=caa2ee2b749042e8b96ebfd4911b111c) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:20.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:20.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f461b-717a-4628-8c5e-8ac702f806df None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:20.590843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 146/294] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:20 2022] GET /identity/v3/role_assignments?user.id=caa2ee2b749042e8b96ebfd4911b111c&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:21.381213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac23b7-31af-4c3f-a25e-57557f1161e1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.381709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac23b7-31af-4c3f-a25e-57557f1161e1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.381783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac23b7-31af-4c3f-a25e-57557f1161e1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.383170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 149/295] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:21.388432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c587b-7b10-4560-8502-5af64ef5e00e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.388846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c587b-7b10-4560-8502-5af64ef5e00e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.389019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c587b-7b10-4560-8502-5af64ef5e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.405765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-dd7c587b-7b10-4560-8502-5af64ef5e00e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:21.407303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dd7c587b-7b10-4560-8502-5af64ef5e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:21.424112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfb96-fd5a-4051-93b3-fb01089d9c4c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.424401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfb96-fd5a-4051-93b3-fb01089d9c4c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.424684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfb96-fd5a-4051-93b3-fb01089d9c4c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.426315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 150/296] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:21.431668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.431954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.432219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.449692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:21.450901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:21.490370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 147/297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:21.496875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3f4b-6cc7-4663-85a9-e92afd5800eb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.496875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3f4b-6cc7-4663-85a9-e92afd5800eb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.496875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3f4b-6cc7-4663-85a9-e92afd5800eb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.496875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 148/298] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:21.498202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.498490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.498767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.516999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:21.518248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:21.538315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 151/299] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:21.538315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.569315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:21.610047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 149/300] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:21.610047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.645912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc212a-03c4-48f3-8765-e97c006ca382 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:21.662545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:21.682191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:21.683183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.683459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.683730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.694264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:21.702458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:21.703310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:21.704083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 152/301] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:21.705356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.713221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fea53-08c0-4538-8ddd-3678893b371f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:21.747361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:21.748093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.748375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.748649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.749800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:21.751275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:21.752875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 150/302] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:21.757653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.801238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:21.814640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764cbd-42e7-4c75-a813-8d77f94b895a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:21.852290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:21.854310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.854830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.855305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.855907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a62cbe-8372-4dda-b27b-882c90b255c0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:21.868513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:21.876409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:21.877271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:21.878168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 153/303] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:21.878809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.882125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:21.894487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:21.895243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:21.895478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:21.895753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:21.896512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:21.896822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:21.898215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:21.911925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 151/304] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:21.911925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:21.998562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7579f4-103a-4fbc-a26b-5c475263072c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.016411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34265753-6702-478b-ba01-1e05c99cff78 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.034303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.035271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.035690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.036099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] PATH_INFO: `/v3/groups` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.036921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.038663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.051085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.051754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.052040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.052321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.053061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:22.053369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.054857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.062923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 152/305] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:22.062923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.109409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:22.119935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 154/306] 10.222.0.16 () {60 vars in 1134 bytes} [Mon May 23 20:04:21 2022] POST /identity/v3/groups => generated 229 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:22.120684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.143491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:22.181306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d89-2af9-42d2-9b44-6c3c1ef8d1fe None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.226103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.227402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.228782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.229198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] PATH_INFO: `/v3/users/4301a248bde04853b331d5fe4c00f484` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.245962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] RBAC: Authorizing `identity:get_user(user_id=4301a248bde04853b331d5fe4c00f484)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.247478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.254468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 153/307] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/users/4301a248bde04853b331d5fe4c00f484 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:22.255475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.271152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d560-4425-40d8-8298-ed96ff20a3e3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.329516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.330515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.330799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.331074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] PATH_INFO: `/v3/users/62c88931434647adb6cab231ca407e69` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.332390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] RBAC: Authorizing `identity:get_user(user_id=62c88931434647adb6cab231ca407e69)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.333974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.336161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 155/308] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/users/62c88931434647adb6cab231ca407e69 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:22.341149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.390626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e26-f378-4a11-9e89-791a25ea4f0f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.393644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:22.433919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.434713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.435059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.435403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.436303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, user.id=4301a248bde04853b331d5fe4c00f484, scope.system=all) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:22.437229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.438746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478db659-0574-4dde-bb1e-188203b0c428 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.444370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 154/309] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/role_assignments?user.id=4301a248bde04853b331d5fe4c00f484&scope.system=all&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:22.447359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:22.464933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab6c-90c4-4870-80b6-af6f725e9b1d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.465231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab6c-90c4-4870-80b6-af6f725e9b1d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.465494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dab6c-90c4-4870-80b6-af6f725e9b1d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.467247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 155/310] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:22.473602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.473889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.474206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.485542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80579d02-4d11-4a30-ba05-daec9f0129f2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.495880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:22.497095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:22.539593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.540812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.541299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.541749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] PATH_INFO: `/v3/system/users/62c88931434647adb6cab231ca407e69/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.543626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62c88931434647adb6cab231ca407e69, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.546165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.575357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 156/311] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:22.598605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.645834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 156/312] 10.222.0.16 () {58 vars in 1334 bytes} [Mon May 23 20:04:22 2022] PUT /identity/v3/system/users/62c88931434647adb6cab231ca407e69/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:04:22.713301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:22.717705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e00648-8588-4cdb-a745-c5ec63c2c995 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.761296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.762155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.762445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.762724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.774912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:22.783045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:22.784031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:22.784808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 157/313] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:22.790322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:22.902891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2ab99-8092-4185-a817-a13a1ab1204f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:22.937927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:22.939027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:22.939488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:22.939938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:22.940910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:22.941406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:22.942993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:22.952191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 157/314] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:22.957878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:23.065905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99876585-d818-41f5-8a1f-6f4838415975 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:23.101965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:23.103173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:23.103461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:23.103735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] PATH_INFO: `/v3/users/f45a681ed7c4464d97e7de8c31e19928` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:23.104946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] RBAC: Authorizing `identity:get_user(user_id=f45a681ed7c4464d97e7de8c31e19928)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:23.106417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:23.108391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 158/315] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:22 2022] GET /identity/v3/users/f45a681ed7c4464d97e7de8c31e19928 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:23.114270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:23.118328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:23.220026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71d77-9a31-4c24-8363-059ae61aa7f1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:23.255682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:23.256336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:23.256618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:23.256894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:23.326422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:23.328461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:23.330351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:23.430789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 158/316] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:23.436234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:23.465236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:23.539474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5af9d-9152-4954-aab6-f5c74e1e56af None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:23.574505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:23.575374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:23.575658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:23.575938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:23.577188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:23.578638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695979f-48a5-402a-979e-4b1b58f1d171 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:23.580684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 159/317] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:23 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:23.586186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:23.689868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47915d8-bab9-4b05-a771-b095de9e87c8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:23.691758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:23.724268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:23.724927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:23.725215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:23.725491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:23.726366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, scope.project.id=9fe28879cfda4d25a493807c5eea929f, user.id=f45a681ed7c4464d97e7de8c31e19928) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:23.726677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:23.728378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6843-7acb-4979-b488-69d53b816063 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:23.735214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 159/318] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:23 2022] GET /identity/v3/role_assignments?user.id=f45a681ed7c4464d97e7de8c31e19928&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:24.494977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe39c1-e7a3-4d46-9d08-22139a8e43d3 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.495463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe39c1-e7a3-4d46-9d08-22139a8e43d3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.495463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe39c1-e7a3-4d46-9d08-22139a8e43d3 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.496805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 160/319] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:24.502135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561673d2-634a-43fc-b136-1ebc3a42ce62 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.502432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561673d2-634a-43fc-b136-1ebc3a42ce62 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.502712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561673d2-634a-43fc-b136-1ebc3a42ce62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.519858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-561673d2-634a-43fc-b136-1ebc3a42ce62 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:24.521046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-561673d2-634a-43fc-b136-1ebc3a42ce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:24.594199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 160/320] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:24.616131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe39c1-e7a3-4d46-9d08-22139a8e43d3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:24.710848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:24.718191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe39c1-e7a3-4d46-9d08-22139a8e43d3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:24.756260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:24.757002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.757283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.757555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.758844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:24.761650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894c34a-4ce8-49e0-98e2-8f06655a2dd8 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.761940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894c34a-4ce8-49e0-98e2-8f06655a2dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.762157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:24.762255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894c34a-4ce8-49e0-98e2-8f06655a2dd8 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.763842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 161/321] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:24.764158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 161/322] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:24.769512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.769974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.770311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.770562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:24.786660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:24.787845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:24.854195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 162/323] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:24.874724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581ba9-522c-4ab5-8687-8e64d7234fa6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:24.876245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:24.904362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:24.912166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:24.912836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:24.913121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:24.913397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] PATH_INFO: `/v3/groups` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:24.914128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:24.915561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367c6bd-4788-42f8-9865-36988c520343 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.001570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa117d66-f125-41f6-9474-d02213652f05 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.021685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 162/324] 10.222.0.16 () {60 vars in 1134 bytes} [Mon May 23 20:04:24 2022] POST /identity/v3/groups => generated 226 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:04:25.023561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df4874-c638-4999-9ad9-e2f723a81320 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.023850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df4874-c638-4999-9ad9-e2f723a81320 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.024134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df4874-c638-4999-9ad9-e2f723a81320 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.025405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 163/325] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:25.035395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.036283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.036570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.036843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.040038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.040618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.040618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.050526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:25.058631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:25.059630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:25.060410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:25.061111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:25.061111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:25.061235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 163/326] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:25.066401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.083284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.137840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 164/327] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:25.161137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.168716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.187433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222b93c-7878-426a-be31-57fb8bd6bde7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.234178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.234849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.235130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.235402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.236145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:25.236447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.237834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.250148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 164/328] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:25.253849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.287088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12fea2-fd7d-4768-9fdf-2bf43895a505 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.334950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.335872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.336321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.336602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.350428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:25.360854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:25.361760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:25.362678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:25.363269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 165/329] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:25.368000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.385085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76104a2-bed7-40f4-aa39-5468198bc7ff None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.386215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.429862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.431330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.431620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.431899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] PATH_INFO: `/v3/users/4301a248bde04853b331d5fe4c00f484` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.433685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] RBAC: Authorizing `identity:get_user(user_id=4301a248bde04853b331d5fe4c00f484)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.435972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.438854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 165/330] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/users/4301a248bde04853b331d5fe4c00f484 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:25.445138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.474559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.493618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46362-a2f6-4818-a946-44d38f127b18 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.532786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.533462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.533777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.534081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.534841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:25.535149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.536601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.545282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 166/331] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:25.550851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.583603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412de9f-043a-4db7-88c9-3c3a23741306 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.632117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.633540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.633951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.634280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] PATH_INFO: `/v3/system/users/4301a248bde04853b331d5fe4c00f484/roles/637edfd3fa00487b93d8c9cb36e578d8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.636489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4301a248bde04853b331d5fe4c00f484, role_id=637edfd3fa00487b93d8c9cb36e578d8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.638215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.657575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.683000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc102c49-8318-42cc-9560-962c6cea7c62 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.703781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 166/332] 10.222.0.16 () {58 vars in 1334 bytes} [Mon May 23 20:04:25 2022] PUT /identity/v3/system/users/4301a248bde04853b331d5fe4c00f484/roles/637edfd3fa00487b93d8c9cb36e578d8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:04:25.726349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.727326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.727605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.728183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] PATH_INFO: `/v3/users/62c88931434647adb6cab231ca407e69` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.730899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] RBAC: Authorizing `identity:get_user(user_id=62c88931434647adb6cab231ca407e69)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.732532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ef412-9675-4301-ad48-8396c7381317 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.734929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 167/333] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/users/62c88931434647adb6cab231ca407e69 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:25.740711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:25.845228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:25.869566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020ff02-58f3-4203-9447-b3d06e5cf3c2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:25.908750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:25.909427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:25.909707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:25.909980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:25.910861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] RBAC: Adding query filter params (role.id=177528ba0fa94e3a9ef5327bad1d58fe, user.id=62c88931434647adb6cab231ca407e69, scope.system=all) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:25.911170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:25.912661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c44007-560a-4dab-a143-a76a08aecf8c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:25.934435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 167/334] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 20:04:25 2022] GET /identity/v3/role_assignments?user.id=62c88931434647adb6cab231ca407e69&scope.system=all&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:27.735461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3fac-e7c7-4dec-9271-d2ec54c845a0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:27.735996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3fac-e7c7-4dec-9271-d2ec54c845a0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:27.735996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f3fac-e7c7-4dec-9271-d2ec54c845a0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:27.737632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 168/335] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:27.747386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:27.747812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:27.748007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:27.771024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:27.772261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:27.849547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 168/336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:27.889206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3fac-e7c7-4dec-9271-d2ec54c845a0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.007924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f3fac-e7c7-4dec-9271-d2ec54c845a0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:28.053838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:28.054699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.054982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.055265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.069348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:28.079193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:28.080114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:28.080888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 169/337] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:28.086565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.166340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:28.213100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024f34d-0123-4426-998a-bb93316e5a39 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:28.260201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:28.260805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.261089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.261364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.262148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:28.262455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:28.263861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98937-11ee-403a-9260-fcc5e26735d8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:28.273239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 169/338] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:28.278692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.310405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:28.392685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a00c50-01d7-45ae-a1ef-192efd4b33fb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:28.428895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:28.429558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.429843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.430148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.505646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:28.507314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:28.509504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:28.546810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a14d2-3e5a-4c6a-9222-a73bb6b1af35 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.547096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a14d2-3e5a-4c6a-9222-a73bb6b1af35 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.547278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a14d2-3e5a-4c6a-9222-a73bb6b1af35 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.548921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 170/339] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:28.554146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.554667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.555159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.593987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:28.595556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:28.616239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 170/340] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:28.622980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.745796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 171/341] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:28.764396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7741d7-148f-468b-9155-362d1749b3ae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:28.769112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:28.787095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.835573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:28.836467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.836817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.837172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:28.842498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:28.844007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:28.846093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 171/342] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:28 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:28.854949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:28.880739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:28.937446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068dc58-e968-450c-b7a4-ba698ad5e899 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:28.984213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:28.985045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:28.985327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:28.985600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.003682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-dc1f0833-6ad9-4aea-aeca-0ba60de8edea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:29.013141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:29.014045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:29.014887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:29.015485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 172/343] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:29.016364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3c51-c147-4438-8267-83bbb0fe3947 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.016649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3c51-c147-4438-8267-83bbb0fe3947 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.016926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3c51-c147-4438-8267-83bbb0fe3947 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.025029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 173/344] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:29.041008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623bfa-5465-4d15-b322-14c6653ff476 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.041292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623bfa-5465-4d15-b322-14c6653ff476 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.041564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623bfa-5465-4d15-b322-14c6653ff476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.062734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da225-5e6f-4844-8fd8-ae19ae0c309d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.066539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-18623bfa-5465-4d15-b322-14c6653ff476 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:29.067934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-18623bfa-5465-4d15-b322-14c6653ff476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:29.124479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.125728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.125837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.126229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.136512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:29.138182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02d40-d2ba-4d7c-aa2c-8ba0a8c20616 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:29.140496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 172/345] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:28 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:29.141919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e2e7e-aa31-4dbd-b6ea-774dd467f968 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.146419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e2e7e-aa31-4dbd-b6ea-774dd467f968 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.146787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e2e7e-aa31-4dbd-b6ea-774dd467f968 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.148588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 173/346] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:29.148842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 174/347] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:29.152948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e2e7e-aa31-4dbd-b6ea-774dd467f968 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.154596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.155106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.156104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.173953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:29.175460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:29.207665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.243703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.267429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 175/348] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:29.268557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.281017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e2e7e-aa31-4dbd-b6ea-774dd467f968 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.317188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.318170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.318597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.319014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.319907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:29.320357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:29.321893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:29.332743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 174/349] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:29.333762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.385041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc3a9-1fa8-400f-9174-ccbaf754a062 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.423030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.423781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.424230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.424745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.425787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=aca8271cb9e64af39cb1d04d5246214d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:29.426315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:29.428046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:29.433963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.435592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 176/350] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:29.436572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.445366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045e9a-1256-4533-ad4a-4e7384002c29 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.488192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.510873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.511709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.511993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.512268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.527619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:29.545983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:29.547185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 175/351] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:29.548203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.561739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa7a6-d907-4514-8b4e-2c1525d4776b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.605554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.606718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.608315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.610359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.633667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:29.651150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:29.652250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 20:04:29.653238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:29.657233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 177/352] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:29.658496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:29.725527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412bab-0224-442b-8b13-9952dd88d58d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.822773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.823523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.823735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.824015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.855772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.860813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717465d-a4ff-494e-a0dd-1ea971ca7db0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:29.865641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:29.906400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:29.906400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:29.906400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:29.906400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:29.906814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:29.906814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:29.907382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:29.915085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:29.916450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 178/353] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:29.917260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c945-0341-4a3c-9409-7c7757827a6d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:29.917375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.049179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 176/354] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:30.049380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd0a28-14a4-429d-a312-807804b6481c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.049465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef30b0-3e40-4a5b-beb7-c83abfafff35 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.049465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef30b0-3e40-4a5b-beb7-c83abfafff35 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.049465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef30b0-3e40-4a5b-beb7-c83abfafff35 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.050114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 177/355] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:30.050852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef30b0-3e40-4a5b-beb7-c83abfafff35 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.091080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.091854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.092162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.092532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.093321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:30.094098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.095039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4b044-1199-4144-8c33-bb512c9c78cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.104194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 179/356] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:30.105461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4fee9-8bf1-4efd-b35b-a3de4c130ae7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.105747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4fee9-8bf1-4efd-b35b-a3de4c130ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.106040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4fee9-8bf1-4efd-b35b-a3de4c130ae7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.107283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 180/357] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:30.107975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4fee9-8bf1-4efd-b35b-a3de4c130ae7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.118511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.140210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.172630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef30b0-3e40-4a5b-beb7-c83abfafff35 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.227089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.227089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.227371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.228318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.236345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4fee9-8bf1-4efd-b35b-a3de4c130ae7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.283849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.284796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.285076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.285350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.286980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.288755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8f22-0654-45f5-9bd2-591a1facdd75 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.291501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 181/358] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:30.292912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8354a-d1c3-4553-989b-d1eb40eb8edd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.293640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8354a-d1c3-4553-989b-d1eb40eb8edd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.293926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8354a-d1c3-4553-989b-d1eb40eb8edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.313722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.314103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5f8354a-d1c3-4553-989b-d1eb40eb8edd None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:30.315208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5f8354a-d1c3-4553-989b-d1eb40eb8edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:30.315309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.384402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.395748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 182/359] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:30.397412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.397854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.398225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.418720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:30.420419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:30.438819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 178/360] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:30.439635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.445014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.521808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 183/361] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:30.522700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.588564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32996963-47cb-44c1-8570-13319dab95cd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.633409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.634339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.634622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.634894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] PATH_INFO: `/v3/users/4301a248bde04853b331d5fe4c00f484` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.636089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] RBAC: Authorizing `identity:get_user(user_id=4301a248bde04853b331d5fe4c00f484)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.637508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.639509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 179/362] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/users/4301a248bde04853b331d5fe4c00f484 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:30.640268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.656608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec590d75-de49-4564-972b-26c3193865e9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.672005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.699918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.700705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.700987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.701262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.702515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.703965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.706870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 184/363] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:30.707634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.743529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.777589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fe4e5-9685-489a-952a-c4a1587dc828 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.822798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.823625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.823907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.824178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.839576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:30.848704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:30.849660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:30.850550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 180/364] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:30.851259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba129c-9003-4df3-9500-ae0a4cc7f1ae None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:30.851335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.891735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:30.892715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:30.893068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:30.893399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:30.894639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:30.896365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:30.898252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 185/365] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:30.898877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:30.957716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:30.980263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a68a3-4f6a-44d2-b00b-a6d888e2b553 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.015596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.024484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55fad1-e60f-4dd3-906d-b67a39b90e60 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.033731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.033731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.033731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.033731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.042274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:31.050057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:31.050942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:31.051721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 181/366] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:31.052352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.073103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.073809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.074188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.074453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.076326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] RBAC: Adding query filter params (role.id=637edfd3fa00487b93d8c9cb36e578d8, user.id=4301a248bde04853b331d5fe4c00f484, scope.system=all) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:31.076642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.078812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.098127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 186/367] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 20:04:30 2022] GET /identity/v3/role_assignments?user.id=4301a248bde04853b331d5fe4c00f484&scope.system=all&role.id=637edfd3fa00487b93d8c9cb36e578d8 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.099248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.176993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88f34-1f5c-44de-86ae-30be7d4ab3c2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.223469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1097ee-018e-48db-ae38-2547cc3139e4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.229197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.230229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.230689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.231136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.232256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=aca8271cb9e64af39cb1d04d5246214d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:31.232805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.234597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.241392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 182/368] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.246795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.262813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.263644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.263978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.264322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.265242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.265773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:31.266891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.268486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.278401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 187/369] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.279503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.296690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.401503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403423d-ef6c-457d-8521-a30a5f66d9f3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.401839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f12a0-1859-4c91-bccf-9ea17414452c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.446462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.446868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.447772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.447877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.448051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.448328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.448729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.449377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.449708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.450570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:31.451086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.451249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.453402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.458164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 183/370] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.459249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.465174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 188/371] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.467457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.566757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.591438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.603852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabb37d-3ce1-427c-95d7-4fc7ca41184d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.632156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a7129-a74e-4833-a91e-2d8f3ea1f155 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.643573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44339e96-1f97-496d-979c-b0272112813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.644515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44339e96-1f97-496d-979c-b0272112813b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.644515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44339e96-1f97-496d-979c-b0272112813b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.644814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44339e96-1f97-496d-979c-b0272112813b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.677586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.678522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.678807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.679488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.680655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=9fe28879cfda4d25a493807c5eea929f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:31.680969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.682900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.692010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 189/372] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:31.692896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:31.783855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44339e96-1f97-496d-979c-b0272112813b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.785338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44339e96-1f97-496d-979c-b0272112813b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:31.819732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea97e7-0e18-48da-9e8e-6fa011f173f8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:31.863170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:31.863170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:31.864415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:31.864415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:31.879493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.958495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:31.993219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:31.995383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f822e15-e4ad-4b11-a8e4-3ac334760179 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.001229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 184/373] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:32.001627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44339e96-1f97-496d-979c-b0272112813b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.141052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 190/374] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:32.145794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44339e96-1f97-496d-979c-b0272112813b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.146521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2abb8-3fbb-4ae4-8c8c-c4476f63aabe None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.146833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2abb8-3fbb-4ae4-8c8c-c4476f63aabe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.147114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2abb8-3fbb-4ae4-8c8c-c4476f63aabe None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.148594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 191/375] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:32.149848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2abb8-3fbb-4ae4-8c8c-c4476f63aabe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.187988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.189676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.190443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.190916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.192321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.194216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6d01c-658b-4d54-84ac-38c15dc61aa2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.197211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 185/376] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:32.198619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.199097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.199564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.199970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.227928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:32.229767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:32.285500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.285914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2abb8-3fbb-4ae4-8c8c-c4476f63aabe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.321300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 186/377] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:32.326356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.339262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.340451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.340928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.341425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] PATH_INFO: `/v3/projects/f5277fd37f8044ad9345367a8cd3028d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.346212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] RBAC: Authorizing `identity:get_project(project_id=f5277fd37f8044ad9345367a8cd3028d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.347895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.350552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 192/378] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/projects/f5277fd37f8044ad9345367a8cd3028d => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:32.356384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.451182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54df945-9587-4e9f-9f45-758802c9ac1a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.476129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a690ad-ba3f-4866-87c4-aa3bbda38a19 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.485244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.505720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.506622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.506902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.507177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.511626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.512694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.515212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 187/379] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:32.518154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.524841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.525988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.526482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.526946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] PATH_INFO: `/v3/groups/c31173e25d394058921362cf297d30b8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.534286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] RBAC: Authorizing `identity:get_group(group_id=c31173e25d394058921362cf297d30b8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.536050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.541406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 193/380] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/groups/c31173e25d394058921362cf297d30b8 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:32.542471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.566215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.630546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a45972-f8e2-4b00-bfc1-d3512249467f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.651316 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db443f-b257-4d95-a210-f8d0451ad4ed None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.669204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.669999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.670315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.670588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.682892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:32.689152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.690084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.690572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.691057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:32.691165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:32.691949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:32.692713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 188/381] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:32.693930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.695030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=9fe28879cfda4d25a493807c5eea929f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:32.695567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.697208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.704456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 194/382] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:32.705585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.745667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.823200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07d29a-a279-4647-86c4-3251732a2f58 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.862108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.863150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.863667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.864897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.865879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:32.866438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.868068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.869524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291fd14-766f-43a9-b1ff-67dc992b6d69 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:32.878533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 195/383] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:32.891612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:32.902445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:32.935404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:32.936864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:32.937513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:32.938146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:32.939620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] RBAC: Adding query filter params (group.id=c31173e25d394058921362cf297d30b8, role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=f5277fd37f8044ad9345367a8cd3028d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:32.940032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:32.942218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:32.951259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 189/384] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/role_assignments?group.id=c31173e25d394058921362cf297d30b8&scope.project.id=f5277fd37f8044ad9345367a8cd3028d&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:33.024074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:33.030311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef986-1b2b-4d0a-9532-1c679fa737d5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:33.084229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:33.085114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:33.085396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:33.085670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:33.086946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:33.088442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:33.090180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 196/385] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:32 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:33.095997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:33.230261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b31ba-e145-45e6-a572-5ffc9fc3f947 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:33.267767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:33.268631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:33.268909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:33.269178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:33.270384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:33.271808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e436b0-6a2a-4f7a-8b75-c0635336a9da None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:33.273491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 190/386] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:33 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:33.279392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:33.411790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b148205-4947-44fc-aa3b-d489933c98a8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:33.448538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:33.449426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:33.449701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:33.449966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:33.452112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, group_id=aaf3397fcda54acfaf9c59f4ce48f3e2, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:33.453563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3500db-416c-4811-b98d-84ed0aa674af None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:33.546708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 197/387] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 20:04:33 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:34.294240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb62b2-c242-4f56-bd6c-bec3db7e1dcd None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.294776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb62b2-c242-4f56-bd6c-bec3db7e1dcd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.294776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb62b2-c242-4f56-bd6c-bec3db7e1dcd None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.296468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 191/388] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:34.301972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f27c8c-3161-40b8-8509-6b5cf5f93070 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.302393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f27c8c-3161-40b8-8509-6b5cf5f93070 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.302580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f27c8c-3161-40b8-8509-6b5cf5f93070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.319325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f5f27c8c-3161-40b8-8509-6b5cf5f93070 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:34.320520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f5f27c8c-3161-40b8-8509-6b5cf5f93070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:34.338774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:34.394128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 198/389] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:34.413215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb62b2-c242-4f56-bd6c-bec3db7e1dcd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:34.462411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:34.508365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3d61-9465-4722-91da-91be26f597d1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.508653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3d61-9465-4722-91da-91be26f597d1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.508848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f3d61-9465-4722-91da-91be26f597d1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.510382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 199/390] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:34.518471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.518984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.519473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.549836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb62b2-c242-4f56-bd6c-bec3db7e1dcd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:34.551078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:34.553941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:34.618704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:34.619865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.620371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.620851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.646078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 200/391] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:34.657038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:34.674048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:34.674995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:34.675784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:34.676379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:34.678601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 192/392] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:34.684143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:34.802470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:34.819459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e8f12-b2e0-4c8f-b3cf-b1dd48171945 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:34.858956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:34.862581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58a2b-b3bf-4302-bf62-983dacc8d649 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:34.874749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:34.875621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.875902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.876177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.895620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:34.903892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:34.904767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:34.905582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:34.909407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 201/393] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:34.911950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:34.930687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:34.931370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:34.931655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:34.931930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:34.932686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:34.932992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:34.934392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:34.954266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 193/394] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:34.955587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.124219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ff910-6811-4cf1-9d75-92ad9ed63d7c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.137931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2361f-4b48-47f8-a6b1-babbe1ed179a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.163584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.164326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.164614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.164890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.165646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:35.166140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.168019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.178683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 202/395] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:35.184559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.195936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:35.217086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.217927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.218416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.221124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.230062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.230062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.230476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 194/396] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:34 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:35.239470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.303688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222da6c3-680e-4826-bec4-cdd9ccc8f9d8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.330354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:35.341385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.342290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.343130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.343475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.344656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.346169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.348718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 203/397] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:35 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:35.354592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.440234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:35.471435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b3eed-7802-40a3-a158-d05c36703616 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.508765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af685e-91b0-4893-9640-8c0fe776d872 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.508955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.509732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.510010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.510326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.511517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.512948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c4b21-15b9-4aeb-bc87-d8235c399c94 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.515179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 204/398] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:35 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:35.516335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92765f6-6628-4227-b05f-bec80501648a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.516614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92765f6-6628-4227-b05f-bec80501648a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.516883 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92765f6-6628-4227-b05f-bec80501648a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.518381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 205/399] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:35.522303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92765f6-6628-4227-b05f-bec80501648a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.557547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.558443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.558722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.558993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.560428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.562783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070e455-0129-4593-93fa-d25121da78b0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.564773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 195/400] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:35 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:35.566455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.566749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.567024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.590272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:35.591960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:35.642406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92765f6-6628-4227-b05f-bec80501648a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.670260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 196/401] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:35.671209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.681940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.682988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.683426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.683787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.686302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] RBAC: Authorizing `identity:create_grant(project_id=aca8271cb9e64af39cb1d04d5246214d, group_id=aaf3397fcda54acfaf9c59f4ce48f3e2, role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.687878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fded2-30d1-4b3b-88c7-2e78c2a75b7d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.698754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:35.757035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 206/402] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 20:04:35 2022] PUT /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:35.758646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d9a3-17db-4635-ab18-e45470057777 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.758924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d9a3-17db-4635-ab18-e45470057777 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.759218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6d9a3-17db-4635-ab18-e45470057777 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.760569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 207/403] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:35.761250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6d9a3-17db-4635-ab18-e45470057777 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:35.772113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:35.782581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c56eae-24e7-46aa-b3ef-7075a4607c13 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.835213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.836588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.836883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.837163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.839559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, group_id=aaf3397fcda54acfaf9c59f4ce48f3e2, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:35.841304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9b99b-19cc-41d0-9750-3f8c9b643f97 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:35.905485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6d9a3-17db-4635-ab18-e45470057777 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:35.941495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 197/404] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 20:04:35 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:35.941495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.941495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.941495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.955203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:35.956014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:35.956344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:35.956622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:35.967156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:35.968799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:35.970422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:35.984122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:35.985208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:35.985990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 208/405] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:35.993499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.052923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 198/406] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:36.065858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.078412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.110037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133486-2654-4b2a-ac81-635c94c10a9e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.118451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.148189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.149298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.149873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.150387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.151359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:36.151892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.153517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.163323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 209/407] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:36.169427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.242800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad0d5-39ad-4ab1-bf52-50b806f4917e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.293582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3ec3-8c88-4b11-a795-ed41c678429e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.302373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.303176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.303457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.303734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.316096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:36.321106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:36.324953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:36.325854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:36.326698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 199/408] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:36.331997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.336896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.337982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.338466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.338939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.340398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.342189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.344347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 210/409] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:36.350494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.430619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.442580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcab8de-41ed-4f6b-8216-ebbade964081 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.457719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2a88-8322-43e6-89c8-96ec08c03cca None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.478798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.479587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.479849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.480217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.481068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:36.481514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.483849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19dec-0afb-4b51-9c41-d01400ad6677 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.495712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.496931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.497538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.497642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 200/410] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:36.498144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.499642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42050-72ac-40fd-b0e7-41c9a1c06dec None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.499928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42050-72ac-40fd-b0e7-41c9a1c06dec None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.500199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42050-72ac-40fd-b0e7-41c9a1c06dec None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.500856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.501762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 201/411] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:36.502481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee42050-72ac-40fd-b0e7-41c9a1c06dec None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.503034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65ed31-f84a-4161-a020-e4956e78d38e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.505285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 211/412] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:36.511686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.512223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.512704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.530387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:36.531961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:36.591527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.604290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 212/413] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:36.605369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.627371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee42050-72ac-40fd-b0e7-41c9a1c06dec None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.672181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.673105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.673448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.673792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] PATH_INFO: `/v3/groups/c31173e25d394058921362cf297d30b8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.675137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] RBAC: Authorizing `identity:get_group(group_id=c31173e25d394058921362cf297d30b8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.676632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.678365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 202/414] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/groups/c31173e25d394058921362cf297d30b8 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:36.683552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.698205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:36.721974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3d3cb-ce9f-4119-ad9e-b2f337f3dd45 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.765283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.766282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.766626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.766984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.769406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9fe28879cfda4d25a493807c5eea929f, group_id=aaf3397fcda54acfaf9c59f4ce48f3e2, role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:36.771161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:36.796352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c02c0-5fb2-4561-9800-da6bd2006d8f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:36.831580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:36.832441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:36.832726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:36.833004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:36.842967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 213/415] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 20:04:36 2022] PUT /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f/groups/aaf3397fcda54acfaf9c59f4ce48f3e2/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:36.846041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.856618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:36.873885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:36.874885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:36.876092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 203/416] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:36.885212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:36.969218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:37.031678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:37.047205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825916ae-ceda-44a0-9bfd-94c85ddb42a2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.097074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6debfc8-178f-4abe-ac17-b23f79b4d081 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.099428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.101104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.101390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.101671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] PATH_INFO: `/v3/projects/f5277fd37f8044ad9345367a8cd3028d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.103182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] RBAC: Authorizing `identity:get_project(project_id=f5277fd37f8044ad9345367a8cd3028d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.104639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.110085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 214/417] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/projects/f5277fd37f8044ad9345367a8cd3028d => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:37.114768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:37.148205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.149054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.149595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.149595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.150396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:37.150711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.152100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.162595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 204/418] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:37.167556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:37.244913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6713fb-3687-431d-be20-6af13b2fa39e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.270439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:37.285430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.286707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.287137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.287578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] PATH_INFO: `/v3/projects/f5277fd37f8044ad9345367a8cd3028d/groups/c31173e25d394058921362cf297d30b8/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.290813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5277fd37f8044ad9345367a8cd3028d, group_id=c31173e25d394058921362cf297d30b8, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.292522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.322238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed28184-a7bb-42a3-8313-2560cbaff399 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.348100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:37.360785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.361677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.362193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.362613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.385655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 215/419] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 20:04:37 2022] PUT /identity/v3/projects/f5277fd37f8044ad9345367a8cd3028d/groups/c31173e25d394058921362cf297d30b8/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:04:37.446997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.448749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.575295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 205/420] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:37.580947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:37.696668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dfdf1-5bdf-40e9-a914-15f34c8c7f87 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.731952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.733612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.734066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.734482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.735816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.737413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.739615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 216/421] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:37 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:37.744845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:37.771303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:37.871074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fcaa-9bb9-4887-b6e3-9adb2fa2fb1a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:37.922748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:37.923630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:37.923920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:37.924201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:37.925498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:37.927001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2c033-c7e2-4b08-b5aa-54ce42608f6d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:37.928678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 206/422] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:37 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:37.934643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:37.998387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:38.056580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732a67-2e71-45e0-9f14-f4645c3add10 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:38.091540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:38.092207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.093010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.093291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.094158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=aca8271cb9e64af39cb1d04d5246214d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:38.094873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:38.097007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d12801-8a20-4cf0-a411-c5ad205349fd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:38.103667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 217/423] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:37 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:38.399367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c4fbe-4cd4-45a1-a4a8-7d64e2b2b5b5 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.399867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c4fbe-4cd4-45a1-a4a8-7d64e2b2b5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.399966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c4fbe-4cd4-45a1-a4a8-7d64e2b2b5b5 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.401530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 207/424] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:38.407151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3b61-9d7d-49dd-b1b1-7d70044f6ef7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.407563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3b61-9d7d-49dd-b1b1-7d70044f6ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.407733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3b61-9d7d-49dd-b1b1-7d70044f6ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.425015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fa6d3b61-9d7d-49dd-b1b1-7d70044f6ef7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:38.426238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fa6d3b61-9d7d-49dd-b1b1-7d70044f6ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:38.505433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 218/425] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:38.530569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c4fbe-4cd4-45a1-a4a8-7d64e2b2b5b5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:38.638732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:38.651831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c4fbe-4cd4-45a1-a4a8-7d64e2b2b5b5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:38.687897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:38.688712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.688993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.689268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.702408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:38.710867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8561d-2a5c-4e48-9abe-e31513d0d928 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.711467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8561d-2a5c-4e48-9abe-e31513d0d928 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:38.711794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:38.712685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:38.717158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 208/426] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:38.718851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8561d-2a5c-4e48-9abe-e31513d0d928 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.720697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 219/427] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:38.721748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:38.726369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.726862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.727310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.747757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:38.749764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:38.830458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:38.839648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 220/428] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:38.867821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811fafb-a6cb-4e9e-9a72-a299b7702eb7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:38.876558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:38.910676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:38.912364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:38.912650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:38.913064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:38.913839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:38.914459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:38.915687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:38.925253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 209/429] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:38.930403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:38.954296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:38.999517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ef80-9c22-420e-b51d-8c12a0059db7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.037583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.038524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.038876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.039267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.051827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:39.057014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92be4c-9354-48d1-9f61-79db7f6c50c4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:39.061948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:39.062856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:39.063646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 221/430] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:39.071775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.109931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.114005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.115006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.115513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.116033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.180067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7d82d-f0b1-43c1-a343-470f1950204d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.192335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.194141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.214261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.219973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.223008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.223008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.223008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.223008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:39.223008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.229869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.242716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 222/431] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:39.250975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.340201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 210/432] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:39.345693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.376504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.387309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b0b49-c7a8-4c04-a768-f4265278a998 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.430882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.432414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.433574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.434000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.466323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe824f-33d9-4197-a29f-7ebafd3653a6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.481812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.503095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.504126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.504413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.504688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] PATH_INFO: `/v3/projects/aca8271cb9e64af39cb1d04d5246214d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.505862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] RBAC: Authorizing `identity:get_project(project_id=aca8271cb9e64af39cb1d04d5246214d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.507456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff9b5c-3a29-4ac0-a234-fd6f42f6d000 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.509579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 211/433] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/projects/aca8271cb9e64af39cb1d04d5246214d => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:39.510932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9da943-8edb-49f4-9b51-9de703274837 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.511304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.511664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9da943-8edb-49f4-9b51-9de703274837 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.512280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9da943-8edb-49f4-9b51-9de703274837 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.514038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 212/434] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:39.514359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d2629-2990-480e-b2c6-23557600d5c3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.514972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9da943-8edb-49f4-9b51-9de703274837 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.613078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.622100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 223/435] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:39.623451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9da943-8edb-49f4-9b51-9de703274837 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.623878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.624167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.624447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.641801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:39.643446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:39.660008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.660872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.661158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.661433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.662635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.664062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.670187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 213/436] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:39.670187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.719806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.734090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 224/437] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:39.734090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.781134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8f1c7-3c94-4494-b02e-257008d445d6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.817351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.818353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.818634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.818902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.820213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.821756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.823587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 214/438] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:39.828137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.847674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca94c5d-860e-4b2b-bc8b-ce67caa537a3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:39.874384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:39.886786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:39.887588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:39.887991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:39.888272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:39.889135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=aca8271cb9e64af39cb1d04d5246214d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:39.889474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:39.891028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:39.897055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 225/439] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=aca8271cb9e64af39cb1d04d5246214d&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:39.897774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:39.962954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104828a-c9af-4169-8dbc-542c41bfe924 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.009579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.013380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.014770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.015063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.015330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.030851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fec4a-035f-4047-ac5f-865b2b47fbbb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.032174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:40.044357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:40.045325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:40.046243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 215/440] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:40.053278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.084047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.085378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.085669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.085949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.087499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.089536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.094664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 226/441] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:39 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:40.097758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.168155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.179570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d626-5658-4508-be66-1aeeec480184 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.217048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.218319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.218612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.219592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.219690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4bf-5842-4793-b489-9aab18b69be1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.219776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:40.219929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.221308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06dda-037e-4f4a-b060-946cb28dc7b0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.231027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 216/442] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:40.234247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b276b8-bf0f-4676-8456-1225be3a80ee None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.234568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b276b8-bf0f-4676-8456-1225be3a80ee None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.235029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b276b8-bf0f-4676-8456-1225be3a80ee None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.238195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 217/443] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:40.238195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b276b8-bf0f-4676-8456-1225be3a80ee None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.262729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.263478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.263835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.265276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.265547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=177528ba0fa94e3a9ef5327bad1d58fe, scope.project.id=9fe28879cfda4d25a493807c5eea929f) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:40.265859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.267341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a63f-2774-4f80-9916-9dc8f16c260d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.273929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 227/444] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=177528ba0fa94e3a9ef5327bad1d58fe => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:40.275616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.275905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.276178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.295518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:40.297096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:40.297961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.382735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 228/445] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:40.391576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b276b8-bf0f-4676-8456-1225be3a80ee None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.411292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.428901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.429477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.429760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.430052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.467600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.513167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.515249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.526370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba73e6-c3d2-4038-ad59-5e6653f3250a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.564301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.566491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.567294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.567574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.567843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.581409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:40.589085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:40.590311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:04:40.591101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:40.591690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 229/446] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:40.597653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.631919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 218/447] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:40.637417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.709396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64136-4b50-4377-b763-5de5b43b744a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.717986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.742948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.743617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.743950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.744206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.745209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:40.745582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.747472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf29bf-e51a-4052-8c24-99f5b3a3065d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.747593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.755975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 230/448] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:40.761439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.782848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.783843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.784124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.784397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] PATH_INFO: `/v3/projects/9fe28879cfda4d25a493807c5eea929f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.785557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] RBAC: Authorizing `identity:get_project(project_id=9fe28879cfda4d25a493807c5eea929f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.787008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.788829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 219/449] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/projects/9fe28879cfda4d25a493807c5eea929f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:40.793802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.804086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.857242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7316-eaf7-44de-9c09-861fa6280ae4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.889826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc82c45-0d47-44cf-b5fe-8e55b8b36adf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:40.893033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.893676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.893958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.894264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.923518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:40.924366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:40.924648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:40.924947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] PATH_INFO: `/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:40.926141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] RBAC: Authorizing `identity:get_group(group_id=aaf3397fcda54acfaf9c59f4ce48f3e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.927555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:40.929144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 220/450] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/groups/aaf3397fcda54acfaf9c59f4ce48f3e2 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:40.932978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:40.934880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:40.960872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:40.962658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:41.008598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:41.030049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729d91-ac93-478c-8b3b-5ef6e9251552 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:41.057956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 231/451] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:04:41.063923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:41.064341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:41.064992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:41.065273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:41.065547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:41.066405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] RBAC: Adding query filter params (group.id=aaf3397fcda54acfaf9c59f4ce48f3e2, role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=9fe28879cfda4d25a493807c5eea929f) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:41.066719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:41.068133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:41.073848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 221/452] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:40 2022] GET /identity/v3/role_assignments?group.id=aaf3397fcda54acfaf9c59f4ce48f3e2&scope.project.id=9fe28879cfda4d25a493807c5eea929f&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:41.134361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:41.167087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d2f7-497e-486b-b3f0-5d640c5d0e42 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:41.202984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:41.203860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:41.204143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:41.204420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] PATH_INFO: `/v3/projects/f5277fd37f8044ad9345367a8cd3028d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:41.205684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] RBAC: Authorizing `identity:get_project(project_id=f5277fd37f8044ad9345367a8cd3028d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:41.207206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:41.209143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 232/453] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:04:41 2022] GET /identity/v3/projects/f5277fd37f8044ad9345367a8cd3028d => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:41.214720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:41.320473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1fea9-119e-49c5-9fc9-223ac3368ee7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:41.353705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:41.354604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:41.354942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:41.355229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] PATH_INFO: `/v3/groups/c31173e25d394058921362cf297d30b8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:41.356393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] RBAC: Authorizing `identity:get_group(group_id=c31173e25d394058921362cf297d30b8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:41.357877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de4a8-5518-4c5b-8848-d6bb818a40d2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:41.360009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 222/454] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 20:04:41 2022] GET /identity/v3/groups/c31173e25d394058921362cf297d30b8 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:41.366029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:41.465895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f76c5-1788-469a-8396-b66da6f99b55 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:41.498945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:41.499589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:41.499872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:41.500144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:41.500965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] RBAC: Adding query filter params (group.id=c31173e25d394058921362cf297d30b8, role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=f5277fd37f8044ad9345367a8cd3028d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:41.501275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:41.502741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4345d4-c930-43ea-a93a-459ab4f29c48 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:41.508326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 233/455] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 20:04:41 2022] GET /identity/v3/role_assignments?group.id=c31173e25d394058921362cf297d30b8&scope.project.id=f5277fd37f8044ad9345367a8cd3028d&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:44.176572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bcaa9-279f-4aaa-acd8-72b9e470f20a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.177090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bcaa9-279f-4aaa-acd8-72b9e470f20a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.177397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bcaa9-279f-4aaa-acd8-72b9e470f20a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.179039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 223/456] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:44.184499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a98aa8-369a-4970-a6f5-592714b1ccfd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.185303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a98aa8-369a-4970-a6f5-592714b1ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.185801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a98aa8-369a-4970-a6f5-592714b1ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.191217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:44.203785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-16a98aa8-369a-4970-a6f5-592714b1ccfd None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:44.205046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-16a98aa8-369a-4970-a6f5-592714b1ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:44.286588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 234/457] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:44.311884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bcaa9-279f-4aaa-acd8-72b9e470f20a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:44.323657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:44.422123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bcaa9-279f-4aaa-acd8-72b9e470f20a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:44.457394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:44.458153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.458438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.458716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.469846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:44.471349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:44.475797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf26b-9ad2-4099-b0e0-6a0f4b45f5fd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.476082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf26b-9ad2-4099-b0e0-6a0f4b45f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.476582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf26b-9ad2-4099-b0e0-6a0f4b45f5fd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.478727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 235/458] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:44.486336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.486868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.487342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:44.495162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:44.496029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:44.496872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:44.496872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:44.496872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:44.496872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:44.496872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 224/459] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:44.502816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:44.507804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:44.509255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:44.579172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:44.582372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 236/460] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:44.607652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:44.623229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29583b-b2a0-4550-a23a-4b5b6313795a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:44.668683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:44.669358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.669643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.669934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.670920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:44.671252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:44.672637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588f939-c3de-44b0-8885-49b93668e715 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:44.676614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:44.688654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 225/461] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:44.694133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a947-c521-4d2e-a97f-9f376bd0d758 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.694133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a947-c521-4d2e-a97f-9f376bd0d758 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.694133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a947-c521-4d2e-a97f-9f376bd0d758 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.694133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 226/462] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:44.695092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654a947-c521-4d2e-a97f-9f376bd0d758 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:44.728094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d57a5-47d7-4d5d-b02a-663cbc70307e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:44.765092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:44.765794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.766103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.766379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.777241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:44.778524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c315b730-5316-4da3-92cf-5a53105a5dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:44.798908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:44.799781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:44.800632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:44.800632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:44.800632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:44.800632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:44.805217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 237/463] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:44.806530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.806825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.807105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.824200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:44.825568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:44.845666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654a947-c521-4d2e-a97f-9f376bd0d758 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:44.874777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:44.894703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:44.895392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:44.895675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:44.895949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:44.896686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:44.899045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a115a-db84-47fc-a79b-f3e5762f3caa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:44.903982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 238/464] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:44.905975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:44.953999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.006941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61fc92-0e3c-401f-9552-3899e71027a1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.021093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 227/465] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:44 2022] POST /identity/v3/users => generated 246 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:45.022669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03536a9e-14e9-454a-99f9-31f97cd2710a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.023013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03536a9e-14e9-454a-99f9-31f97cd2710a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.023297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03536a9e-14e9-454a-99f9-31f97cd2710a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.029384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 228/466] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:45.029384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03536a9e-14e9-454a-99f9-31f97cd2710a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.056669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.058516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.059065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.059544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.060591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:45.061658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.064724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865e26-ee48-46ea-92df-277e7863f0ab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.083265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 239/467] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:45.085079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.085586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.086084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.109483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:45.111491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:45.169420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03536a9e-14e9-454a-99f9-31f97cd2710a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.210338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 240/468] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:45.210338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.242356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.243183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.243515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.243799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.254478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.256908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.258488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:45.278165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:45.279273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:45.280266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:45.280266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:45.280266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:45.280266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:45.280266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 229/469] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:45.280738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.313872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.341269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdfedb-341b-4a52-a6e7-558f3ae9f495 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.378734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.379418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.379702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.380206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.380684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.382233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.404906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a848f18-e07e-416d-aced-0fc646cf4189 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.442297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.443115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.443400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.443674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.457329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.458871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.465840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 241/470] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:45 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:45.472256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:45.494873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:45.495803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:45.496658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:45.496658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:45.496658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:45.496658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:45.496658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 230/471] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:45.502705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.641968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.664017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3238c-e50c-4f1a-9083-1d05a4348bd1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.669342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.682923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31471623-0336-4a6d-8e21-a38680bf4248 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.702390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25771853-d114-471c-8938-24c6c2372488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.703335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25771853-d114-471c-8938-24c6c2372488 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.703619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25771853-d114-471c-8938-24c6c2372488 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.703901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25771853-d114-471c-8938-24c6c2372488 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.704798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25771853-d114-471c-8938-24c6c2372488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:45.705112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25771853-d114-471c-8938-24c6c2372488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.706848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25771853-d114-471c-8938-24c6c2372488 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.722346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 231/472] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:45.726276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.727159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.727592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277743a-9038-46f1-8130-c7b8f5d4b170 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.727748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.727891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277743a-9038-46f1-8130-c7b8f5d4b170 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.727993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.728150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277743a-9038-46f1-8130-c7b8f5d4b170 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.728719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:45.729026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.729529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 232/473] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:45.733937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24edd2-24a4-4116-a7df-5a38e9787cb5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.734084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277743a-9038-46f1-8130-c7b8f5d4b170 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.745133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 242/474] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:45.746477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.746769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.747047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.767522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:45.768734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:45.848109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 243/475] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:45.849367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:45.869765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277743a-9038-46f1-8130-c7b8f5d4b170 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:45.908772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:45.909572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:45.910081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:45.910488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:45.911562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:45.913297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:45.947287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.948942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:45.969011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eacbc-6f16-4234-b1e1-e3660a2cddc3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:46.004761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:46.005445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:46.005728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:46.006005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:46.006764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:46.008223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:46.023199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 233/476] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:45 2022] POST /identity/v3/users => generated 249 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:04:46.024258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:46.105223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 244/477] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:45 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:46.148358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4769b9-3523-4096-8f31-4833fd997193 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:46.196170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:46.196780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:46.197063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:46.197337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:46.198119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:46.199513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:46.247836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.sql.core [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 20:04:46.247836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 20:04:46.247836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [parameters: {'id': '28aca88530944ba9be2a347301418821', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 20:04:46.247836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 23 20:04:46.265210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return ref.to_dict() May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:04:46.266166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application yield resource May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.rollback() May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.force_reraise() May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise self.value May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application session.commit() May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 20:04:46.267103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._prepare_impl() May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.flush() May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._flush(objects) May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application compat.raise_( May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application flush_context.execute() May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rec.execute(self) May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 20:04:46.267956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:04:46.268905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:04:46.269719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [parameters: {'id': '28aca88530944ba9be2a347301418821', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:46.272751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 20:04:46.274738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:46.278512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 234/478] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 20:04:46 2022] POST /identity/v3/roles => generated 144 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:46.278654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:46.393566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e764-44d1-4b72-a077-65555ca1809e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:46.431499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:46.432719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:46.433203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:46.433674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:46.436900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:46.454135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-79ca967c-339c-474f-a6d8-bdfcb8d4c9fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:46.455586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:46.457137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 20:04:46.458683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:46.460747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 245/479] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:04:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:46.465835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:46.532634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:46.577276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb918d-befc-410f-845d-8b8a3a0cf763 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:46.612992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:46.613862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:46.614274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:46.615079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:46.615951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:46.616366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:46.617844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59b713-8071-4a4a-9aa3-c19ddb457017 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:46.626806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 235/480] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:47.512654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286011-9982-426a-b668-f9b1b9d61f3c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:47.513342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286011-9982-426a-b668-f9b1b9d61f3c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:47.513342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286011-9982-426a-b668-f9b1b9d61f3c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:47.514856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 246/481] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:47.519695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:47.520169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:47.520324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:47.537371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:47.538580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:47.619000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 236/482] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:47.653954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286011-9982-426a-b668-f9b1b9d61f3c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:47.790402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286011-9982-426a-b668-f9b1b9d61f3c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:47.836573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:47.837374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:47.837650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:47.837920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:47.854149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:47.862557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:47.863442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:47.864233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 247/483] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:47.869774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:47.958858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.024825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fae66d-ff17-43c5-a71c-9f4c93c5ffbe None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.068648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.069258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.069550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.069836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.070625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:48.070939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:48.072347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4898b5e-b7d1-4a7b-a298-5cc44ec57157 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:48.085259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 237/484] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:48.091084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.193550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.209491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258394e-6803-47d8-83cb-313d95458617 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.209783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258394e-6803-47d8-83cb-313d95458617 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.210100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258394e-6803-47d8-83cb-313d95458617 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.217725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 238/485] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:04:48.223113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.223402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.223684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.244525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:48.245717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:48.277926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad4190-6d18-4e8c-8cfc-79066ad29040 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.315744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 239/486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:48.339258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.367599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.368406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.368780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.368951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.389496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.393744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:48.397598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:48.438873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:48.439820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:48.440659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:48.440659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:48.440659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:48.440659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:48.440659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 248/487] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:48.445359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.482357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee309a-6c56-4430-a6d5-34e2427ca2aa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.519956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.520842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.521135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.521413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.534352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:48.542453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:48.543402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:48.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 240/488] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:48.549920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.666042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4c2d-c82e-4a05-adec-7426a2860725 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.666839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.692743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.693428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.693715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.693995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.694838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:48.695199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:48.696813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:48.710087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 241/489] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:48.712279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.716076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cdf96-61aa-4559-b700-9d648a5abff5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.752093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.783200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.783879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.784530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.785059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.786092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:48.786596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:48.788495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce6fb8-95c0-4dce-8f34-037961a8d0af None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:48.801947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 249/490] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:48.803559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6549d-da22-4a51-ba91-df5fc8b1f0ba None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.803944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6549d-da22-4a51-ba91-df5fc8b1f0ba None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.804318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6549d-da22-4a51-ba91-df5fc8b1f0ba None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.810186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 250/491] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:48.811898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6549d-da22-4a51-ba91-df5fc8b1f0ba None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:48.841534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b2a-84c8-4bac-a16c-2a2af5d5e9fb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:48.878164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:48.879367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.879487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.879785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.891580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:48.893293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-cb8f2410-56f0-44e5-b3e6-0258ded07e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:48.911818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:48.912889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:48.913825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:48.913825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:48.913825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:48.913825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:48.919223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 242/492] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:48.919223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:48.919223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:48.919223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:48.938393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:48.939721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:48.955524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:48.958797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6549d-da22-4a51-ba91-df5fc8b1f0ba None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.002245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.003042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.003325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.003603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.011037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.012533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.026072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 243/493] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-46e3f81a-3f11-4797-962c-3daf05925fbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:49.028041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:49.028922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:49.029868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 251/494] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:04:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:49.031827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.034768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c918-5e6f-40ce-9ece-578091f151f9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.035052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c918-5e6f-40ce-9ece-578091f151f9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.035326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186c918-5e6f-40ce-9ece-578091f151f9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.040629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 252/495] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:49.041665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186c918-5e6f-40ce-9ece-578091f151f9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.072405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.138411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c26ba-7731-48c8-877a-9b99b686b204 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.150798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186c918-5e6f-40ce-9ece-578091f151f9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.173938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.174622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.174909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.175183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.175910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:49.176220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.177596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acbe32-5585-458b-8d9d-b7f96c076c9e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.187973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.188745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.189033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.189304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.190171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:49.190478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.191916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.192465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 244/496] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:49.194215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.194692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.195127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.214295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:49.215814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:49.228635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 253/497] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:49.228922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.229713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.303622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 245/498] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:49.311835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.318260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.333575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe04bdf-ad65-4eea-8d55-c11baef5e7a1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.368675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.369474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.369761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.370052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.380239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:49.388078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:49.389020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:49.389798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 254/499] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:49.390932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.428273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbf087-b3f6-4538-a081-0983f2405bcb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.462460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.463514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.463960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.464373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.472163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.473771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.475720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:49.491395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:49.492341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:49.493235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 246/500] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:49.493780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.503845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12b48-26ef-42cd-b506-de6aea07e297 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.537699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.538402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.538686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.538960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.555213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.599772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1acc47-4cc7-4087-93f2-11bd5aba7d3e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.616880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.618397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.636653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.637461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.637748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.638052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.650318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:49.658603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:49.659485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:49.660251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 247/501] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:49.660897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.724647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.726153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 255/502] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:49.731390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.773852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6244b6-104e-4df1-82c3-c65901171f52 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.781985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.807445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.808113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.808397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.808673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.809417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:49.809724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.811225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.819558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 248/503] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:49.820354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.834900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd0abd-0630-46e7-ad1d-b6165e2fe634 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.868244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.868907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.869178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.869461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.870340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:49.870646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.872081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.889404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 256/504] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:49.890182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.918943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bafad-b211-4dc4-9c5e-4635c8734759 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.938697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:49.952981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:49.953647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.953930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.954230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.954983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:49.955311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:49.956663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2eb6b-9194-4303-a1e3-706ac14e63f2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:49.964664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 249/505] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:49.965854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51397-9be0-4bea-bbd0-debae79c79e0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:49.966168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51397-9be0-4bea-bbd0-debae79c79e0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:49.966442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51397-9be0-4bea-bbd0-debae79c79e0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:49.968652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 250/506] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:49.969695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51397-9be0-4bea-bbd0-debae79c79e0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:49.992620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f62ab-339d-47ba-b53a-10fc0605673d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:49.994930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.026149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.026953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.027239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.027514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.038313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.039777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:50.056833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.057702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:50.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:50.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:50.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 257/507] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.058913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.071362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51397-9be0-4bea-bbd0-debae79c79e0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.106417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.107212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.107500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.107783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.117567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.119066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:50.136549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.137386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.138268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:50.138268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:50.138268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.138268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:50.138268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 251/508] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.138528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.163659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e4481-6267-41ec-8716-af73eb353892 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.174634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.197002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.197648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.197936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.198285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.223481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.239460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fb888-4a49-49c3-82e7-23b97a0672e0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.266116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.267532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.273014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.273809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.274124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.274402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.284179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.285616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-95ac305b-c415-48d5-8f8f-b3fbe4e37a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:50.302759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.303615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.304447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:50.304447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:50.304447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.304447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:50.304447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 252/509] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.305092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.305375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.305650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.322162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:50.323333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:50.364689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 258/510] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:50.365513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.388146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 253/511] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:50.389485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.415428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.427997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.470334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4abe-db7a-4ff0-8e4e-c1d2cbd4fe94 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.494319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca4399-ccd9-48f5-b24e-0da886111ca6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.503942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.504601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.504904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.505161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.505882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:50.506218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.507605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.519619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 259/512] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:50.520332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.531511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.532170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.532449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.532719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.533431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:50.533736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.535121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.550601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 254/513] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:50.550601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.615322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d89a3d-0427-4c85-aebb-ab8dcb05bf1e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.621649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.631463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.643792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3e32-dc1a-48c9-80d3-2b951e09f74b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.648064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.648716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.649025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.649294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.650004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:50.650336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.651696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.662397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 260/514] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:50.663157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.679817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.680635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.680910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.681183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.692135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.693622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:50.710252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.711111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.711948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:50.711948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:50.711948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.711948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:50.711948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 255/515] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.712215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.764829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ab8bd-8166-4f63-8f78-51100c1f3514 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.797392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.798152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.798437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.798713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.808864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.810333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.814142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847a2f-c476-4ec7-83e9-f948d488c536 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:50.827333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.828171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.829038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:50.829038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:50.829038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:50.829038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:50.829038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 261/516] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.829296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.841498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.849670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:50.850509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:50.850789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:50.851060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:50.858271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:50.859695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:50.861760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:50.874845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:50.875741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:50.876572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 256/517] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:50.877036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:50.986996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7e0f-b90c-4c5e-aa25-2198a0aa1a98 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.018852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.019641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.019770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c568e27-75fb-49d9-ad5c-1c6ac1363062 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.019926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.020197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.027022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.028464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:51.043163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:51.044007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:51.044872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 262/518] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:51.046591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.055643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.056736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.057023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.057613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.064630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.066069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:51.080301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:51.081311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 257/519] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:51.082612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.132592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.132812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.144253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb233ca-42c7-4617-a9f6-99ae608357a2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.177208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.177867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.178185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.178465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.179182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.179490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.180971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.181095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5829baf-8b2a-4ad0-b9f9-7e955caf6e3c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.191795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 263/520] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.192689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.216093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.216761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.217563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.217837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.218598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.218905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.220275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.232234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 258/521] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.233519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.292663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc4f-f43b-41e9-b4db-d881b51f140f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.324262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.324847 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.325129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.325398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.326697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.327006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.328440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.332487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0641c-fbfa-4bd7-a238-4c8b319de51a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.341393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.345625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 264/522] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.346455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.357025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.366551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.373078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.406951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 259/523] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.407762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.441645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bedf79-d635-464d-a0a6-367926ea8431 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.474978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.475643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.475924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.476199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.477064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.477638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.479112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.495481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 265/524] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.496238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.505147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e10484-4a0f-46b1-be31-6ed4c7128c8a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.540802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.541601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.541880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.542189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.548603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.550073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.558852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:51.563999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:51.564829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:51.565683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 260/525] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:51.566164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.593618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc90c1-f67f-4202-9378-75aeb32dbd14 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.596956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.628014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.628675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.628956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.629229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.629929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.631404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.667139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb618f4-37b9-45f3-9936-eefb853ab4b9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.699232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.699878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.700162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.700439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.701880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:51.702227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.703638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:51.708925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 261/526] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:51.709768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.759786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 266/527] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:04:51 2022] POST /identity/v3/users => generated 247 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:04:51.760932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:51.843148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.861608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95d399-2ea8-4e29-b0d2-9cb4cc1b75fb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.899834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b6034-ab8e-4af4-8f99-4e44147b3779 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:51.906058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.906678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.907074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.907447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.930056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:51.945962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:51.946678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:51.946965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:51.947244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:51.996106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:51.997561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.039965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.041456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.104820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.117007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 262/528] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:52.118408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.160186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 267/529] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:52.161049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.185188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.229219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b7d34-2314-4637-b9ed-62d8bd617599 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.266811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56e0f2-8e9a-450f-89e1-79dc787037bc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.267642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.268566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.268983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.269479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.270467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:52.270914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.272481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.292092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 263/530] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:52.293125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.303456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.304392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.304786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.305159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.316255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.317855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:52.335670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:52.336722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:52.337759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:52.337759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:52.337759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:52.337759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:52.339406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 268/531] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:52.340615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.352044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.408689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848b84-c651-451b-a09f-740cb2eb8c65 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.418204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.444059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36842703-f0be-475a-a49e-cb4b98e96b62 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.446510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.447463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.447881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.448312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.460522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.462275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.480677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.481486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.481888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.482288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.483779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=bef73ca7ffe94b79b085dcfa1eae278d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:52.484192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:52.484824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:52.485792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:52.486755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:52.486755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:52.486755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:52.486755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:52.487268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.488882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 264/532] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:52.489938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.494365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 269/533] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/role_assignments?user.id=bef73ca7ffe94b79b085dcfa1eae278d&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:52.494521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.600463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465ecbc-ac23-44a6-a259-884b1e3b2204 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.604075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213e9c1-1f9b-4cd5-aa28-07148e50a199 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.614253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.648317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.649018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.649365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.649714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.650442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:52.650756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.653622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.666876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:52.677668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 270/534] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:52.680743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.682250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.718895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 265/535] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:52.722837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.857955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbcb6a-c129-43b4-9ee1-37aeee591fce None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.894599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208255d-a878-4acb-9c83-09d26e9a6aae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:52.924417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.925208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.925488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.925771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.933067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.934614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:52.952058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:52.952953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:52.953776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 271/536] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:52.959336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:52.966538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:52.967900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:52.968181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:52.968455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:52.981159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:52.982696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:53.017043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:53.017936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:53.018810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:53.022990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:53.026624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 266/537] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:53.034508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:53.067527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bce31c-1021-47a7-a0d9-60115bb0137f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:53.091587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:53.107411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:53.108093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:53.108379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:53.108661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:53.109541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:53.109860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:53.111697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:53.126174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 272/538] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:53.132924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:53.267629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33d77d-9744-4a6c-a4a7-f3d1a3ee10a3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:53.315418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:53.317337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:53.317760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:53.318867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:53.319800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:53.320790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:53.322532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:53.324438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3096d6-27ed-45e3-96b7-657b0e27f67e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:53.344846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 267/539] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:53.350856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:53.352964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:53.387772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:53.389361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:53.389822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:53.390292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:53.391290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=567602cb67c447c798d4009f77140b85) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:53.392858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:53.394503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b6f9-140f-41bf-9f78-db7ec1a1ef4f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:53.403739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 273/540] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:53 2022] GET /identity/v3/role_assignments?user.id=567602cb67c447c798d4009f77140b85&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:53.464203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:53.483848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848df-7e6a-4ced-8775-55529efd4a2b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:53.519819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:53.520876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:53.521333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:53.521824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:53.522867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=617e5a9c04ec4d5fa7833ab46264142d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:53.523353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:53.525071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837eefa-9c2b-4021-bf9f-a8e7076fbcdb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:53.532060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 268/541] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:53 2022] GET /identity/v3/role_assignments?user.id=617e5a9c04ec4d5fa7833ab46264142d&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:54.183971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0efed-5e23-4152-acd9-4c320946ad58 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.184478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0efed-5e23-4152-acd9-4c320946ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.184551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0efed-5e23-4152-acd9-4c320946ad58 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.186072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 274/542] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:54.187526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c3def-6d20-4a31-8df7-35d55cd46bb7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.187896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c3def-6d20-4a31-8df7-35d55cd46bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.188080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c3def-6d20-4a31-8df7-35d55cd46bb7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.189687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 269/543] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:54.194852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.195140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.195414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.201982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.202290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.202570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.219597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:54.220807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:54.239322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:54.240523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:54.331338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 275/544] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:54.357410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:54.359440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 270/545] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:04:54.411760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:54.518455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:54.519148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c20c9-b60d-4509-8473-92f02cac7e6e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:54.544035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:54.562664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81e69-fec9-4e65-8db4-d418f4d97490 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:54.588160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:54.588978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.589255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.589523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.612279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:54.620077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:54.620968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:54.621742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 276/546] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:54.629211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:54.630767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:54.631579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.631858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.632131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.652842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:54.660731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:54.661601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:54.662399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:54.663958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 271/547] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:54.675029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:54.759750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc76ba-810b-429e-8406-59c2e446ead1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:54.795644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22656a7-8788-477f-9163-66487779426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:54.798731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22656a7-8788-477f-9163-66487779426a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.799183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22656a7-8788-477f-9163-66487779426a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.799601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22656a7-8788-477f-9163-66487779426a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.800498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22656a7-8788-477f-9163-66487779426a None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:54.802046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22656a7-8788-477f-9163-66487779426a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:54.803577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22656a7-8788-477f-9163-66487779426a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:54.822600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 277/548] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:54.828777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22656a7-8788-477f-9163-66487779426a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:54.881474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953152-a93a-4875-95c4-9cdd11fa019b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:54.911264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:54.961156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:54.961836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:54.962154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:54.962431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:54.963315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:54.963473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:54.964891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:54.971539 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:54.983917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 272/549] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:54.995205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.018455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22656a7-8788-477f-9163-66487779426a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.061834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.063190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.063472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.066366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.086447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.087912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:55.118089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:55.119180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:55.120226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:55.120226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:55.120226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:55.120226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:55.120226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 278/550] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:55.128102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.157819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df1782-0bd7-4ba8-b8f1-608078be4d2f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.201677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.202481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.202761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.203205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.214171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.215816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f25b2b52-acb4-425f-9dd7-2ff61aff5773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:55.235596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:55.236783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:55.237834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:55.237834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:55.237834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:55.237834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:55.243956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.244475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 273/551] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:55.246121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b487a4-2e8e-4a8c-bc6a-a27c74166a2f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.246559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b487a4-2e8e-4a8c-bc6a-a27c74166a2f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.247007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b487a4-2e8e-4a8c-bc6a-a27c74166a2f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.249584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 274/552] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:55.255004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b487a4-2e8e-4a8c-bc6a-a27c74166a2f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.262144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbefe4-792e-4852-a1a6-3a70d1c9cf58 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.297920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.299171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.299452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.299730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.300522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:55.301234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.302283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59928f9a-84a2-4771-b62d-60e1e10aeb27 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.313035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.315163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 279/553] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:55.316421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.316706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.316981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.333743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:55.335089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:55.369877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b487a4-2e8e-4a8c-bc6a-a27c74166a2f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.414038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 280/554] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:55.414038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.415714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.416576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.417006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.417430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.418351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:55.418824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.420932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.439600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:55.446448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.491197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.513989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7e7c9-a1c8-430e-a1d3-92bbe2109f86 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.549902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.550811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.551202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.551348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.554184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b205d42-8c2b-4aec-845c-c843a81135f0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.558900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.560469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.566527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:55.575063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:55.576461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 281/556] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:55.577639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.595855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.596904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.597338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.597786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.608027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.609678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:55.624277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:55.625459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:04:55.626995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:55.629132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 276/557] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:55.630078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.689143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ea277-156e-4402-a759-fd7b83624988 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.724124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.724929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.725210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.725484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.729359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.736620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b1c32-068e-4e1b-8fd7-411b45e769ed None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.741717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:55.751396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:55.752344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 282/558] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:55.758326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.782254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.783212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.783680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.784139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.785179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:55.785671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.787455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.805904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 277/559] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:55.807121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.847390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:55.886996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8154-da1a-4fb8-bd3e-9ca6e204af74 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.929327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.930090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.930376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.930678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.931552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:55.931860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.932758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2fbf8-c455-4e5d-ba60-da6099fed63a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:55.933321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.953723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 283/560] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:55.959154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:55.971097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:55.972102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:55.972586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:55.973060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:55.974056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:55.974559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:55.976156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:55.989809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 278/561] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:55.991031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.019660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.066676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de251d-fcea-4e3d-b718-a6d73ecd6bb3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.092548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.104045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8b375-771c-44c3-905f-2325720b7355 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.107441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.108365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.108654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.108928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.121382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.123476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.141767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.142453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.142735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:56.142834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.143734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.149274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:56.149274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:56.149274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.149274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:56.149508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.149580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 284/562] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.149580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.218041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.219917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.257066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d1e80-1b4d-4527-9e41-a0f2c09a05e9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.265249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.292038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.292826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.293107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.293381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.303362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.304798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.313440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b5357fe8-5e94-4ab4-8070-9e30bbd9ca47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:56.322655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.323633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.324522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:56.324522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:56.324522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.324522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:56.324522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 285/563] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.325181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed141db-bbba-4c3a-80c1-9371e3e65171 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.325467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed141db-bbba-4c3a-80c1-9371e3e65171 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.325740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed141db-bbba-4c3a-80c1-9371e3e65171 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.326833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 279/564] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:04:56.330061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 286/565] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:56.330061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ee9-6cb3-465e-b9a8-af5ad4060e5f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.330061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ee9-6cb3-465e-b9a8-af5ad4060e5f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.330061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf0ee9-6cb3-465e-b9a8-af5ad4060e5f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.330389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.331082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 287/566] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:04:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:04:56.334770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf0ee9-6cb3-465e-b9a8-af5ad4060e5f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.427487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643df0f-9036-4d72-a66d-dd9cfda12f2d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.431966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf0ee9-6cb3-465e-b9a8-af5ad4060e5f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.461627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.462318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.462602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.462880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.463604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:56.463919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.465274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.465559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.466371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.466683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.466962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.476713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 280/567] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:56.476840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.477756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.478446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-08888c2f-3007-4572-be15-9ca68d0eaf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:56.496745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.497824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.498846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:56.498846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:56.498846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:56.498846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:56.498846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 288/568] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.499163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f245-d79c-42dc-a781-d67e255e83ec None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.499447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f245-d79c-42dc-a781-d67e255e83ec None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.499728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3f245-d79c-42dc-a781-d67e255e83ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.515859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-faa3f245-d79c-42dc-a781-d67e255e83ec None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:56.517039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-faa3f245-d79c-42dc-a781-d67e255e83ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:56.520176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.527365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.575382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee6423-d49c-4b84-902d-c62c29816bf0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.582099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 289/569] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:56.583440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.583736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.584022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.607948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:04:56.609151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:04:56.613429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.614184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.614466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.614738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.615458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:56.615760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.617109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.629011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 281/570] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:56.629774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.678879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 290/571] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:04:56.679888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.725148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a73a4-0a94-40be-b88e-5d62a0334f0a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.736151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.755743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.759197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.759994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.760283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.760556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.767000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.768439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:56.782382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 282/572] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.789957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.790649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0a49a-4744-46df-8227-34001ed620e9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.818983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.819641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.819923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.820199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.820915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:56.821219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.822636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.833443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 291/573] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:56.834262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.886246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b54e-9fe1-4fb5-8bc2-51b94ce41951 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.920519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.921312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.921596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.921872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.932002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:56.932124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d732d-0f73-4fa4-95f6-e80b518af9a9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:56.939788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.940760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:56.941604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 283/574] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.943163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:56.963654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.969679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:56.970570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:56.970848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:56.971124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] PATH_INFO: `/v3/users/glance` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:56.977781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:56.978645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:56.980082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:56.994481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:56.995455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:56.996391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 292/575] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:56.996991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.046168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ab67-7ba7-4e37-834a-248308fc1501 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.078032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.078825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.079104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.079377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.089266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:04:57.094468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9210c-8f37-4d58-a07a-eb75f77302ee None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:04:57.097467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:57.098464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:57.099300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 284/576] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:57.100933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.129226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.129880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.130196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.130501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.131282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:57.131589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.133045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.144541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 293/577] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:57.145307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.193295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.195702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.201969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef966b9-d86b-4926-893f-9475e262882c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.235658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.237239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.237586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.238072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.244885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70a352-82e1-42b2-82dc-3bdc9b2266f9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.245042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.246390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:57.260200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:57.261188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:57.262080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 285/578] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:57.262596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.283516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.284187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.284473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.284747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.285494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:57.285805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.287223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.295046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 294/579] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:57.295810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.363093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e1c7-a75d-4302-af85-2ca4bf882a18 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.394626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc07ca-574a-42d3-b6f3-f1e7845831c9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.395413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.396087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.396375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.396666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.397497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:57.397810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.399259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.407357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.416381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 286/580] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:57.417063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.427263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.427452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.428118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.428401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.428677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.429418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:57.429726 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.431134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.439289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 295/581] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:57.440065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.516192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881350-cfdd-406b-8600-8c55cbd4f278 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.537551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385487-aa25-4168-9cda-a888b5038239 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.548620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.549606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.549892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.550199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.564418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=4cf871584be84949a82a9da23fb16153, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.565854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.569377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.570040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.570334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.570604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.571376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:57.571678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.573088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.585405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 296/582] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:57.586223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.618211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.661592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 287/583] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:57 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:57.662697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.681891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ddcfa-c34c-4543-9e28-c8438c4c6469 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.722288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.723267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.723733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.724101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.736287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.737428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.737772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:57.756618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:57.757737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:57.758678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:57.758678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:57.758678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.758678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:57.758678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 297/584] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:57.759030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.790651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1230e7-9fab-4006-b746-597dadde9550 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.830173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.830994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.831277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.831553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.845798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.847577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.855870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:57.868792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:57.869892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:57.870793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:57.870793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:57.870793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.870793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:57.870793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 288/585] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:57.871071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:57.872465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35162e8-438b-4606-a6a7-19cf196d3c70 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:57.907811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:57.908601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:57.908890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:57.909170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:57.919124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:57.920625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:57.938265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:57.939216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:57.940083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:57.940083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:57.940083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:57.940083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:57.940083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 298/586] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:57.940375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.017176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b42301-abc0-4206-bfd3-d24ceeebc35f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.028071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.052164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3c745-bff9-4518-bfae-76f5222c8385 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.052345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.052709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.052995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.053270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.054246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=1cc42255281f442bb146fe386f432cc2) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.054555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.055972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.074390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 289/587] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/role_assignments?user.id=1cc42255281f442bb146fe386f432cc2&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.074390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.113526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.114264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.114986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.115265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.116276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.116582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.118350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.128344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.136855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 299/588] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.138086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.205159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2cf0-3603-4593-8961-8ae1fb14a470 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.251416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.252116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.252375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.252651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.253392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.253701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.255134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.272225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 290/589] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.272430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847520d-4ed6-4db8-9e9a-9c663279464c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.273120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.309512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.321237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.324992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.324992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.324992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.324992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.324992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.326283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.348182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 300/590] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.349531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.382868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a2271-bc85-4372-b1f5-52c2ee412f61 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.410704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.421821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.422730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.423094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.423325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.430990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.432457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:58.447695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:58.448671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:04:58.449632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:58.453571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 291/591] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:58.454518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.459673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa51a2-90ac-4e4f-97b7-5d52936b3366 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.495997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.497148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.497649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.498175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.505298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.507023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:58.520896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:58.521796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:58.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:58.524831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 301/592] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:58.525820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.563290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.570478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd4861-0a97-4953-9932-1f81684f4d65 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.615183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.615998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.616284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.616563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.625084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.626966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.633157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bde44-4dfb-4ae1-8cb4-0c9e78905649 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:04:58.642409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:58.643336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:58.644267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 292/593] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:58.645414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.650894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.670448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.671130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.671411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.671673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.672508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.672817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.674503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.697649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 302/594] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.698795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.755937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abccfc0-2b88-4d05-aee1-88451ca225a7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.795194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.796238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.796672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.797195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.798185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.798663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.800472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.805911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac4fe-a9ad-41e7-8031-3b4f967b0ad3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.814394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 293/595] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.815516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.832858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.857448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.859048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.859523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.860004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.862007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:58.862359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.864399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:58.889436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 303/596] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:58.890582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:58.924087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25d89a-5074-4e31-964c-e0a6537cb612 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:58.935038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:58.962603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:58.963633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:58.964084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:58.964500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:58.977201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:58.978900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:59.000523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:59.001558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:59.002564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:59.002564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:59.002564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:59.002564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:59.004301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 294/597] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:59.005206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.025064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002473cd-e1c9-4f57-a783-0250a9b97ea5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.071889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.072712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.072995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.073272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/bef73ca7ffe94b79b085dcfa1eae278d/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.074973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:59.096013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=bef73ca7ffe94b79b085dcfa1eae278d, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.097459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.117490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da390b32-b30a-408a-b1eb-939756c788da None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.152279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.153070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.153346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.153614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.164358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.165779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.178038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 304/598] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:58 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/bef73ca7ffe94b79b085dcfa1eae278d/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:04:59.178038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:04:59.200830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:59.202637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:59.203595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:04:59.203595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:04:59.203595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:04:59.203595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:59.205425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 295/599] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:59.209495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.340293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:59.360536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f6133-b21b-40c8-83f4-bd093e4c9f01 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.409136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.410087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.410511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.410926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.412102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:59.413380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.415139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.437009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 305/600] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:59.443318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.453458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32139865-ce12-41b9-b0d9-8039c2836d99 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.476761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:59.496120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.496797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.497082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.497360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.498662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:59.498980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.500367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.514962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 296/601] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:04:59.522063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.630672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e4b5c-bd80-45a6-a645-a36e26921007 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.635476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff20e6-ca9f-43c9-a57e-4f91d553a8bb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.666215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.667217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.667578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.667927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.675772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.677633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:59.692639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:59.693725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:59.694824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:04:59.696677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 297/602] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:04:59.698733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.699701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.700082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.700453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.702609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.719695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.721319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:04:59.752642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:04:59.754236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:04:59.756309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:04:59.762967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 306/603] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:04:59.767608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.776393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:59.781994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:04:59.854559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586166-d381-48f3-beaf-80171edebdd7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:04:59.896473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:04:59.897073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:04:59.897357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:04:59.897634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:04:59.898504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:04:59.898817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:04:59.900267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:04:59.916704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 298/604] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:04:59.926382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:04:59.975650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d72e-86db-4d67-854f-0854600c3eb5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.017943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.018684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.018967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.019242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.020133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:00.020446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:00.022004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846531-d04e-4b9d-bc60-bbeb2d334b2a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:00.035553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 307/605] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:00.036831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411b86-c7bf-496f-80f1-46ca8f88fbcd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.037118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411b86-c7bf-496f-80f1-46ca8f88fbcd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.037395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb411b86-c7bf-496f-80f1-46ca8f88fbcd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.043607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 308/606] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:00.044410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.045658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb411b86-c7bf-496f-80f1-46ca8f88fbcd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.051149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944051d-5d61-44a9-b9ac-5e888e72b894 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.086384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.087284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.087567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.087845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/617e5a9c04ec4d5fa7833ab46264142d/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.104223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=617e5a9c04ec4d5fa7833ab46264142d, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:00.105796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe634e40-0228-4e59-a5e6-07349dfd1c3e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:00.123948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.163561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb411b86-c7bf-496f-80f1-46ca8f88fbcd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.198848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 299/607] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:04:59 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/617e5a9c04ec4d5fa7833ab46264142d/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:05:00.200509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf0ac-c8ea-4b78-840f-d7de0c3cd35d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.200802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf0ac-c8ea-4b78-840f-d7de0c3cd35d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.201079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbf0ac-c8ea-4b78-840f-d7de0c3cd35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.209712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.211052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.211546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.212107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/567602cb67c447c798d4009f77140b85/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.235090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-59dbf0ac-c8ea-4b78-840f-d7de0c3cd35d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:00.235333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=567602cb67c447c798d4009f77140b85, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:00.236876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59dbf0ac-c8ea-4b78-840f-d7de0c3cd35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:00.237119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:00.318450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 309/608] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:00 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/567602cb67c447c798d4009f77140b85/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:05:00.329703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 300/609] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:00.374575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.427826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd24b4c-d990-406d-abe2-ddf966dfdae8 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.428184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd24b4c-d990-406d-abe2-ddf966dfdae8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.428427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd24b4c-d990-406d-abe2-ddf966dfdae8 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.430055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 301/610] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:00.452315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.452675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.452949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.491655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:00.493762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:00.543001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c20a49-a0dc-4f5e-aa9f-4ddaf8f42241 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.566225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.590562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.591556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.592364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.592637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.593750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 302/611] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:00.603485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:00.611343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:00.612328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:00.613659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 310/612] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:00.618681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.624855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.641276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.737017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541fb15-9579-4ae0-8305-23f6d07e16e6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.737423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750d0a9-ce87-47de-8e6c-eb3b4daff9e2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.773020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.773698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.773978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.774280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.775123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:00.775431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:00.782124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:00.788478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 303/613] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:00.790462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.791535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.792029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.792570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.796831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.811218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:00.828820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:00.829797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:00.830665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:00.833447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 311/614] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:00.839444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:00.872069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.881461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:00.906561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb1db7-8c72-4ef7-bc9a-76315781ee16 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:00.944014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:00.945057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:00.945513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:00.945941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:00.960572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:00.962381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:00.981481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:00.983214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:00.984311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:00.984311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:00.984311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:00.984311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:00.986353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 304/615] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:00.992133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.033741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c30101-e9ac-4eb5-a801-cae1bda4b94d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.098661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc305f4c-6989-4120-8e19-c968908cc2a4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.109077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.109761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.110059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.110341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.111219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:01.111388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.112755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.123974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:01.133385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 312/616] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:01.137246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.138186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.139180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.139479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.139761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.140815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:01.141285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.142825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.156245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 305/617] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:01.161702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.268900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0cd41-8f0c-4571-9811-d3d2e4caa977 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.347673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:01.360937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574f3ae-6850-4555-b49d-d52ceaf86079 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.409339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.412199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.412199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.412199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.425495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.427000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:01.461475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:01.462617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:01.463503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:01.463503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:01.463503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:01.463503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:01.469096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 313/618] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:01.473640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.479701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.480508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.480788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.481059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.502822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.504262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:01.531592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:01.532556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:01.533488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 306/619] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:01.538790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.630193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:01.635496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6515-eb09-4123-936d-aefd2f0f12a0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.695903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.696850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.697305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.697727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.698666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:01.699123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.700673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.710447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68a989-2d9b-42a2-891c-9bb909dfd0b5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.726549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 314/620] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:01.733388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.747723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.749128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.749508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.749880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.751100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:01.752174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:01.752743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.754461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:01.774833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 307/621] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:01.788063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:01.894885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:01.898753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bac30-de00-49f6-aeb4-4de15daff367 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.936488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee21fa4-5381-4ad2-b45c-f4ba0bf8f23a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:01.937747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.938827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.939348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.939816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.971539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:01.972380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:01.972616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:01.972895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:01.980836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:01.983146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-762531cb-2a5b-4b20-b81e-c9927d8c4f64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:02.000587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:02.001596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:02.002570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 315/622] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:02.003767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126534-4dad-477e-b123-686f924303de None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.003767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126534-4dad-477e-b123-686f924303de None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.003921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126534-4dad-477e-b123-686f924303de None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.005404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 316/623] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:02.010995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.011278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.011555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.022366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.023996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.031462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:02.033246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:02.115954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.126141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 317/624] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:02.127181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.135153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.143021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 308/625] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 20:05:02.144506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.270097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a537d00-03c1-4bf5-873e-58998c667894 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.270358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa3ba-6ae6-4808-9e7f-a0167073da9b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.314272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.314526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.314979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.315258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.315533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.315937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.316361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.316471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:02.316673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.317031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.318336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.329601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.331778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.343205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 318/626] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:02.344517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:02.356122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:02.357800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:02.358775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:02.358775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:02.358775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:02.358775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:02.359962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 309/627] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:02.360172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.415448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.431010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.468317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be0460-36a3-4b60-ba92-09558545ff57 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.492022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59bf9-9b9f-4ae0-ab48-eabc20d29e0e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.508831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.509874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.510292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.510682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.522591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:02.531531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:02.532465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:02.533369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 319/628] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:02.534095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.539251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.540612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.541075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.541552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.555853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.557948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:02.580767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:02.581727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:02.582649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:02.582649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:02.582649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:02.582649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:02.582649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 310/629] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:02.583701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.663382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e30e5-3528-4f76-bbd9-12aa7f0cdf5c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.682573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.697924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ebbb-f3fb-4b13-bccb-679c84c3d02e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.702417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.705537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.706604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.707106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.707575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.708523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:02.709056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.710930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.725841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 320/630] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:02.727339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.741616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.742563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.742849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.743128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.743884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:02.744193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.746364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.757489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 311/631] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:02.758689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.866864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03bafa-e0bf-47d4-b509-d4bdb0a8654f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.877431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be071a-20f3-4779-b307-43f7267f7aa4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:02.909379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.910067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.910373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.910652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.911375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:02.911845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.913175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.913742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:02.914575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.914855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.915133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.922085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:02.923501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:02.929398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 321/632] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:02.929398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-78b03634-69f1-4502-9730-0ffca764608e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:02.944207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:02.945317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:02.946233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 312/633] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:02.946745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a462f-bec3-4b70-b0e1-6837d9e96937 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:02.946813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a462f-bec3-4b70-b0e1-6837d9e96937 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:02.947198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a462f-bec3-4b70-b0e1-6837d9e96937 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:02.953520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 313/634] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:02.954299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a462f-bec3-4b70-b0e1-6837d9e96937 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:02.972813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:02.981642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.040308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3e3ec-0724-4c80-8733-fa716a385a52 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.056106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a462f-bec3-4b70-b0e1-6837d9e96937 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.072431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.073274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.073555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.073836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.084921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.086456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.090453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.091239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.091544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.091815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.098367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.099758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:03.103504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:03.104587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:03.105611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:03.105611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:03.105611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:03.105611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:03.105611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 322/635] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:03.105968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0cf11268-2555-4bba-bf7c-5c03074aa852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:03.114378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:03.115425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:03.116703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 314/636] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:03.117390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fe95-76d9-41bf-b90b-50e611488109 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.117390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fe95-76d9-41bf-b90b-50e611488109 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.117550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fe95-76d9-41bf-b90b-50e611488109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.133912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a616fe95-76d9-41bf-b90b-50e611488109 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:03.135111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a616fe95-76d9-41bf-b90b-50e611488109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:03.197871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.199187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 315/637] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:03.200550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed96e-f38a-4f13-aa17-9a385048adfb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.201435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed96e-f38a-4f13-aa17-9a385048adfb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.201667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed96e-f38a-4f13-aa17-9a385048adfb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.203097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 316/638] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:03.203700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ed96e-f38a-4f13-aa17-9a385048adfb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.218124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e02c-1154-4756-8d50-ee6c2ae45fe2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.225444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.253323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.254102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.254385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.254657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.255446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:03.255761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.257187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.268427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 323/639] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:03.269199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.309178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ed96e-f38a-4f13-aa17-9a385048adfb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.342262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.342919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.343203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.343486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.344205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:03.344515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.345883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068dc50-29e2-4070-8fb3-96f98b252866 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.357599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 317/640] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:03.359008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.359282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.359560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.366428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ea3de-a74f-4652-bc0c-a06e3ba08fe0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.376365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:03.377522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:03.400966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.401607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.401886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.402195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.402967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:03.403270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.404696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.416391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 324/641] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:03.417765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.428133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.445631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 318/642] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:03.446564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.454679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.523547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad328f4-0a5e-422d-9326-65652a0144f6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.542149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f6e5f-9608-4953-9f8d-82ad856b9067 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.556916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.557705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.557989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.558293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.568929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:03.574591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.575235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.575515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.575841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:03.576635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:03.577759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:03.583845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:03.584535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:03.584535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.584678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 325/643] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:03.584748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.584822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.584892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 319/644] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:03.584892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.655674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.666347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.694143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb55da-19c5-41b4-84cc-f389dd0150b1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.700152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0e85-aff5-4dc6-b92e-4d8949bb1c45 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.731673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.733392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.733713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.733942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.740075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.740988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.741278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.741552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/1cc42255281f442bb146fe386f432cc2/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.742612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.744091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.758109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=1cc42255281f442bb146fe386f432cc2, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:03.759551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:03.759653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:03.760493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 20:05:03.761565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:03.766004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 326/645] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:03.766329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.830585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 320/646] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:03 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/1cc42255281f442bb146fe386f432cc2/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:05:03.831805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.877550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1bbbf-2df9-419b-b85f-baf53215c184 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:03.909930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:03.921916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:03.922796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:03.923071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:03.923350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:03.935679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:03.943877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:03.944870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:03.945694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 327/647] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:03.946426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:03.990097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4447-ec55-4ca0-ad3f-1fb591c3e735 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.020142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.038370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.039128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.039405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.040231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.040545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:04.040860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.042259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.051401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 321/648] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:04.052277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.066198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8226e-9683-4521-98ff-ead8db7ff247 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.112242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.112919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.113202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.113484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.114919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:04.115235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.116949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.140387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 328/649] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:05:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:04.141237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.173708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583904-29ea-426e-ba94-ef5b39ac349e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.194418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.227415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.228386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.229130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.230319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.231091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:04.231408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.232826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.246556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 322/650] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:04.247092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.271190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2a845-a8e0-4bee-b5ec-265ff3dcfe89 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.296360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.319729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.321675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.322208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.322690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.338540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.340887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.357258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c0bd6-25b2-4240-aa6a-25e96ef6992d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:04.365402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:04.366305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:04.367231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:04.367231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:04.367231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.367231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:04.369787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 329/651] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:04.370781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.399966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.400623 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.400901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.401173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.471192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.472665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.499252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.505487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782b73-db84-4d38-85a9-9a6a688b9056 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.518462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.543609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.544712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.545144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.545567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.556684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.558456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.575011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 323/652] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:04.576546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:04.579187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:04.580078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:04.580950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:04.580950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:04.580950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.580950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:04.583907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 330/653] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:04.584873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.678601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152f36-2a7f-4224-86d9-a21a15e68e24 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.693774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc86a-f7e4-4093-837e-6ff0ea424f69 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.713564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.714380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.714728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.715073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.715860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:04.716228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.717681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.729916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 324/654] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:04.730847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.734757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.735672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.736735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.737212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.737666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.749762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.751522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:04.770911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:04.771827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:04.772711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:04.772711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:04.772711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:04.772711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:04.775047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 331/655] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:04.776124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.779252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.836078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7ac5-aeec-4705-98d5-14798a98384e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.874648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.875670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.875785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.876045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.876772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:04.877078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.878611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.889156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97918efe-1bbc-40a6-ab4f-f0d71b8c3c2a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:04.893579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 325/656] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:04.894724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:04.926977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:04.928264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:04.928848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:04.929324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:04.939209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:04.941066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:04.956955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:04.957937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 20:05:04.959208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:04.962006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:04.962270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 332/657] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:04.963935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.003683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be09b3-ed9b-4b88-95e9-2b3e4e97fcc6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.029627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.041679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.041899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.047099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.047099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.047099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:05.047099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.047099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.059101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 326/658] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:05.059600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.078790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bbd3f-3f77-4551-9cf4-3b743ad0fe5b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.117120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.118383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.118891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.119432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.128311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.130295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:05.145715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:05.146770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:05:05.147708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:05.150577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 333/659] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:05.151677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.164176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5508-600d-4182-bafb-f1168c4b27cd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.189887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.201080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.201852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.202177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.202455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.203286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:05.203612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.205016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.224315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 327/660] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:05.225195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.272620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38cdc9-7509-421a-bc94-9d497e483177 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.298584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.311119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.314755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.314755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.314755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.320545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.322528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.331215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818d488-b051-49d9-b648-7a5680e5aea0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:05.338640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:05.339497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:05.340448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:05.343721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 334/661] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:05.344729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.367864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.368536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.368817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.369091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.369921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:05.370270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.371796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.390315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 328/662] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:05.391296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.434751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.456811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0662-b380-46d9-bdfc-b94307bef234 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.495318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.496322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561514-9d8a-442d-abf2-a32309dd74d4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.496575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.497074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.497591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.533532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.534173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.534470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.534728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.535513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:05.535818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.537339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.550239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 329/663] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:05.551075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.555620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.576764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.578833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.655388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.658754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a532f7-3dc0-406e-a444-850ab860822c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.694503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 335/664] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:05.695428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.697110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.697869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.698185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.698788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.771583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.773250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.781250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.801067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b7692-2359-4b3a-89e5-fc47071e19bc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:05.836670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:05.837432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:05.837723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:05.838003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:05.838861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=bef73ca7ffe94b79b085dcfa1eae278d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:05.839172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:05.840717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:05.847961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 336/665] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/role_assignments?user.id=bef73ca7ffe94b79b085dcfa1eae278d&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:05.848897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.884952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:05.890552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 330/666] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:05.899182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:05.962485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db15d-0a9a-41b0-ad1c-f55b1df58f12 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.004410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.005221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.005504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.005778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.021864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.023391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.031969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a2562c6f-85bb-4763-8e42-d16606a2c854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:06.044857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.051677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 337/667] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c849d-b810-4367-b368-00e1cdbc298e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c849d-b810-4367-b368-00e1cdbc298e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c849d-b810-4367-b368-00e1cdbc298e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.052590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 338/668] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:06.054248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c849d-b810-4367-b368-00e1cdbc298e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.103261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37736fc-8031-44c3-9877-0c442e45de94 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.146368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.147219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.147503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.147781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.158696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.160252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.170888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c849d-b810-4367-b368-00e1cdbc298e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e756721c-7389-4c5a-8e8c-4ae9176d447a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:06.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.181835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.182989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:06.182989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:06.182989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:06.182989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:06.182989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 331/669] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.184090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335a3d3-655c-4d17-9954-56bca7d24f54 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.184090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335a3d3-655c-4d17-9954-56bca7d24f54 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.184244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335a3d3-655c-4d17-9954-56bca7d24f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.203911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7335a3d3-655c-4d17-9954-56bca7d24f54 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:06.204971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7335a3d3-655c-4d17-9954-56bca7d24f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:06.217218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.217893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.219111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.219448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.220307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:06.220612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.222164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.239050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 339/670] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:06.240317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.269603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.288447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 332/671] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:06.289636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa891-7c98-49f0-960e-93f8a105512d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.292789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa891-7c98-49f0-960e-93f8a105512d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.292789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa891-7c98-49f0-960e-93f8a105512d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.296279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 333/672] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:06.296822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa891-7c98-49f0-960e-93f8a105512d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.315939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.361095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6983ac8-d7c2-4b04-87ff-c09520812082 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.397029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.397694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.397971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.398271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.398991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:06.399292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.400687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3a3b7-d4ea-4ef5-944d-e6097e49f039 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.401001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa891-7c98-49f0-960e-93f8a105512d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.416759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 340/673] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:06.416759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.416759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.416759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.430648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:06.431832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:06.443303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.444171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.444453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.444725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.452405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.453874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:06.468677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.469569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:06.470514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:06.474690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 334/674] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.475370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.502156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 341/675] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:06.503123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.533938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.554653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.598295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcab0e0-b5b4-42eb-92e9-c0ebbaddd092 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.605233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82b7d-cb73-4219-8ffb-ff706b9fdeb2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.636599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.637429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.637715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.637991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.650136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:06.658640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.660074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:06.660934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 335/676] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.662274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.765717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.766566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.766850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.767127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.773690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936f7cf-31b9-486a-88af-9d2cdfae7cce None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.774061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.775204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.775587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:06.790129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.791057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:06.792017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 342/677] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.792504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.817505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.818205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.818489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.818767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.819556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:06.819871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.821299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:06.833873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 336/678] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:06.834935 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.901148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7cb0-f0c1-4570-bf73-9e722b7472b7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.911298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:06.935791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.936588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.936872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.937160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.947841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:06.948973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b0d4-3108-4045-be57-03ac056dabb5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:06.955564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:06.956497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:06.957339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 343/679] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:06.959032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:06.990079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:06.990974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:06.991401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:06.991769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:06.992615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:06.993050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:06.994550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 337/680] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.032854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.065800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387f97f-dc66-46ec-a181-5c94ab2ccb97 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.104268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.105039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.105437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.105714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.106529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.106855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.108396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.111571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f939d-2753-493d-a5af-506b83c04aa9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.122413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 344/681] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.123688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.148543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.149218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.149501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.149775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.150656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=617e5a9c04ec4d5fa7833ab46264142d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.150973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.152508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.155193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.166054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 338/682] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/role_assignments?user.id=617e5a9c04ec4d5fa7833ab46264142d&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.166054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.268594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13159-a90a-4013-92c1-f602fb4ff63c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.276337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.290511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230a872-4180-4497-b36c-fca463abbcd9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.320284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.321245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.321530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.321805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.322857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.323163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.325315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.329028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.329952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.330357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.330722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.335968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 345/683] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.336908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.350040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.352537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:07.376640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:07.377778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:07.379030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:07.379030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:07.379030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:07.379030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:07.379030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 339/684] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:07.379521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.456819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.467053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5ee9c-6624-4afd-80f2-4f92b0759cb4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.503115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.504148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.504624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.504930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90000adb-d7c5-4c37-8e09-b9b24ba127ef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.505312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.506373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=567602cb67c447c798d4009f77140b85) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.506878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.508497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.515631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 346/685] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/role_assignments?user.id=567602cb67c447c798d4009f77140b85&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.516744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.578745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.579938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.580414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.580879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.600496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.609613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.611595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.638926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e1d13-80df-4c4e-9cdd-7b1bc9b1d1cf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:07.660225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:07.661493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:07.662394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:07.662394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:07.662394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:07.662394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:07.662394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 340/686] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:07.668804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.701449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.702165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.702444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.705786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.706595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.706958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.712748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.739916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 347/687] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:07.745947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.774491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:07.847012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef4f2f-4bf1-4a1a-8660-e4e8186644d7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.890328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f206c-89d3-4614-bd9f-34c4d4c2b888 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:07.895211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.895894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.896181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.896463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.897202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:07.897514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.899417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.919942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 341/688] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:07.921529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.940433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:07.941611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:07.941899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:07.942221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:07.952222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:07.954240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:07.973926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:07.975022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:07.975960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 348/689] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:07.982788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:07.991822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.070299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815d05d-9b5c-42f6-87d2-e3be2f0c6ae8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.085554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.114837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.122196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.122196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.122196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.126491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.128080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:08.144077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:08.145083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:08.146002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 342/690] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:08.149337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122471-65ea-48da-be1f-3c8968a6851f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.165015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.194699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.195710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.197374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.197850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.200253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:08.202055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.203700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.247450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 349/691] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:08.248333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.331365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4255d0a-d7ba-4011-b60b-485c7340e151 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.348115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.375689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.376369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.376652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.376928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.378170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:08.378484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.380626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7a841-d06f-414c-a253-ffaf8be709a2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.390055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a616f-08d9-4906-843d-7645b7715898 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.403239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 343/692] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:08.405565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48ee9-aec6-47b4-9998-807cdcef3329 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.405854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48ee9-aec6-47b4-9998-807cdcef3329 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.406155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db48ee9-aec6-47b4-9998-807cdcef3329 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.414663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 344/693] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:08.415125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db48ee9-aec6-47b4-9998-807cdcef3329 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.433413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.439052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.440035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.440607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.441102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.514927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.516300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc556de4-c9b7-44ff-9147-1c89f9bad547 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.547021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db48ee9-aec6-47b4-9998-807cdcef3329 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.591223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.592371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.592652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.592925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.622564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 350/694] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:08.623931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.624219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.624489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.639477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.642876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:08.644066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:08.672813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.674282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.678371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.732346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 351/695] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:08.733687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.780239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 345/696] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:08.785232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.843708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6f20-ea74-483c-ad2a-7044892bb143 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.858396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.879812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.880910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.881436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.881926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.891232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d71f65-a39e-4fe9-93ea-7b3dc5182da4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:08.893609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.895296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:08.913186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:08.914132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:08.915328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:08.915328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:08.915328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:08.915328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:08.917140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 352/697] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:08.918143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:08.925902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:08.926743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:08.927024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:08.927357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:08.935512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:08.940291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:08.941955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:08.959887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:08.960838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:08.961778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:08.961778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:08.961778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:08.961778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:08.968520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 346/698] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:08.969608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.022303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7d85c-c25d-4560-a761-524561cd13a5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.058178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.059128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.059254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.059633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.070527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.072080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.084144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267991e-7c9f-48a8-8994-f3b60818e5d9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:09.090475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:09.092243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:09.093177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:09.093177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:09.093177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:09.093177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:09.096229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 353/699] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:09.097081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.118830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.135507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.136568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.137081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.137558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.139093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:09.139643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.141313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.159242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 347/700] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:09.160439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.178516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.199177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a5950-9c8f-4290-b6dd-03fb016a81a8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.234591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.235261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.235537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.235809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.236919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:09.236919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.238256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.250282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 354/701] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:09.251288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.273069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0552d1b-7fa6-4c0b-bdbd-e9414149dafe None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.310685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.311759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.312249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.312783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.313717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:09.314272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.315990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.332889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 348/702] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:09.334006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.353761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85510-5d14-4a64-94f2-09256cb52675 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.380459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.381569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.391446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.392617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.398213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.398213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.401515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.403234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:09.418552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:09.420256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:09.421407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:09.428285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 355/703] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:09.429337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.448472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8dd07-b5ff-4d55-80e2-61142b4722f1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.484616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.485522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.485844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.486247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.487064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.488690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.533636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c76cf5-cdf1-49a2-958d-a526e8c6e1b7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.568951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.569671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.569992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.570280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.577178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.578677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.590359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 349/704] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:05:09 2022] POST /identity/v3/users => generated 254 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:09.591939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e6c2fd9e-2cb4-4a41-a8b7-1626d1d31ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:09.592567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:09.593412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 356/705] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e5bf6-6851-4472-95de-b8a73fd32dbe None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.599238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e5bf6-6851-4472-95de-b8a73fd32dbe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.599864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e5bf6-6851-4472-95de-b8a73fd32dbe None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.599864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 357/706] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:09.604212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.604615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.604890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.624641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:09.626086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:09.692904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.722346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.728896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2a6d3-0855-42d2-966e-c6e5172077ba None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.734947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 358/707] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:09.736212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.788336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.789428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.789918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.790425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.791423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:09.791983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.793756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.809056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 350/708] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:09.810290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.871266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57990ef4-94e4-4908-a0a7-c49340590a94 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.911295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.912405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.912888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.913358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.914399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:09.914904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.916535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:09.929156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8322d1-693b-4ce9-911f-655238ccc635 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:09.930363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 359/709] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:09.931441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:09.987283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:09.988483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:09.988966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:09.989443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] PATH_INFO: `/v3/services/network` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:09.990395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:09.990543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:09.992299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:10.007329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:10.008624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:10.009523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.010299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 351/710] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:10.011432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.038212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:10.068689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553ee5a-e6b3-476e-9b15-ee1c5fbe5cbb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.108174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.109189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.109755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.110269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.111287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:10.111877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.113672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.115401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd53b6-5e1e-45ec-85d7-3620f6c2b8ea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.120698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 360/711] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:09 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:10.121776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.179262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.180487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.180772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.181109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.182052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=1cc42255281f442bb146fe386f432cc2) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:10.182392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.183881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21335427-2a7f-48d6-9f63-a0b138d47160 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.193922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 352/712] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/role_assignments?user.id=1cc42255281f442bb146fe386f432cc2&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:10.202219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c30a4-a58d-4c28-9f52-522d481ed0b7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.202219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c30a4-a58d-4c28-9f52-522d481ed0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.202219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c30a4-a58d-4c28-9f52-522d481ed0b7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.202528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 353/713] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:10.204681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.204973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.205254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.248018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:10.249297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:10.293952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:10.300852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc2c0b-04fe-4bb3-9cc8-606918309b92 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.355265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.355956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.356240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.356517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.357348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] RBAC: Adding query filter params (name=network) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:10.357665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.359146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.364951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 361/714] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:10.375077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.457963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 354/715] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:10.487858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728d490-26f1-412b-9f58-462e73765406 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.494026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.523770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:10.530103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.531193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.531624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.532092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.532948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] RBAC: Adding query filter params (type=network) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:10.533414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.535182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.542466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 362/716] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:10.618412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31643043-d2d3-466b-82e4-de845dc581fd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.665752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.666547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.666830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.667108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.667779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.669285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:10.682786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:10.683763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:10.684595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 355/717] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:10.690765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.726067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:10.808976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805834b9-ad3e-4b40-a990-10ccd5bdc640 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:10.853403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:10.854104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:10.854386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:10.854658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:10.855429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:10.855733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:10.857659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e72d3-6be4-4a4e-a188-5e22db7b3940 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:10.869483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 363/718] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:10.879124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:10.997449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:11.020975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0420-6496-4720-ad40-e4fad5116762 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:11.062532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:11.063211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:11.063497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:11.063776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:11.064492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:11.064804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:11.066207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f7b7-4c66-40b7-9bbb-b8d43e77eeba None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:11.072279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 356/719] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:12.291570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606571-f1d7-4c59-be5f-99cdf00003df None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.292537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606571-f1d7-4c59-be5f-99cdf00003df None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.293046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1606571-f1d7-4c59-be5f-99cdf00003df None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.294910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 364/720] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:12.300295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.300686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.300869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.327595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:12.328819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:12.443748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 357/721] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:12.478283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1606571-f1d7-4c59-be5f-99cdf00003df None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:12.572442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:12.582381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1606571-f1d7-4c59-be5f-99cdf00003df None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:12.617372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:12.618195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.618477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.618747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.629922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:12.638608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:12.639538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:12.640366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 365/722] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:12.647608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:12.727528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:12.755576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46be9e-b80d-4ac5-b53f-558d3ce2675d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:12.807598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:12.808280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.808564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.808836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.809644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:12.810494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:12.811846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711c130-5ebd-4b0f-9fae-387f96743d0d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:12.820322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 358/723] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:12.827298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:12.934437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4aae6-722b-49a2-ac97-39c51172ba2f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:12.959914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6218-083c-4048-9c9b-8812f184fb20 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.960519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6218-083c-4048-9c9b-8812f184fb20 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.961014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6218-083c-4048-9c9b-8812f184fb20 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.962958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 359/724] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:12.968186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:12.968775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:12.969250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:12.987991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:12.989575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:13.002952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.004139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.004871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.005347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.029035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.031700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:13.052530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:13.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:13.054416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:13.054416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:13.054416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:13.054416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:13.054416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 366/725] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:13.062991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.067581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.140807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 360/726] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 20:05:13.162717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.231443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d721-7040-4ed0-94ca-f65b78587b82 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.258670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.279522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.280425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.280712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.280988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.281743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:13.282268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.285271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269017b-dbc9-4db8-9bd9-32839827dda9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.300495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062433-ce96-4333-adf0-f837d58a53f8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.304074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 367/727] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:13.305456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a308b-abb2-4b1f-a74d-37231166fe86 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.305743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a308b-abb2-4b1f-a74d-37231166fe86 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.306030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a308b-abb2-4b1f-a74d-37231166fe86 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.312340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 368/728] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:13.314166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a308b-abb2-4b1f-a74d-37231166fe86 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.343755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.344634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.344915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.345232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.358218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-01eaa67e-9c1c-47a9-92a3-6c8de4ce5b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:13.365909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:13.366902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:13.367784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:13.369727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 361/729] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:13.371353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.371839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.372866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.394508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:13.395689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:13.407923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.448691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a308b-abb2-4b1f-a74d-37231166fe86 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.483150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 362/730] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:13.483999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.493386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.494227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.494506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.494790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.504746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.506509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6b70b62c-d5ae-4ce1-a7dd-1648c4dfa00a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:13.522293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:13.523345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:13.524897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 369/731] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:13.525528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a542875-eb69-4547-9a6f-e722696e2288 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.525528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a542875-eb69-4547-9a6f-e722696e2288 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.525874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a542875-eb69-4547-9a6f-e722696e2288 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.532887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 370/732] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:13.532887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a542875-eb69-4547-9a6f-e722696e2288 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.550550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.595399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec3533-baba-4091-87ff-4949856ea59e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.630999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.631686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.631940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.632210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.632951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:13.633255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.634819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.639598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a542875-eb69-4547-9a6f-e722696e2288 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.647733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 363/733] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:13.647733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.667298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.678602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f9934-9c14-4881-befa-61ef571919da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.679392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f9934-9c14-4881-befa-61ef571919da None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.679744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f9934-9c14-4881-befa-61ef571919da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.680028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f9934-9c14-4881-befa-61ef571919da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.689903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f9934-9c14-4881-befa-61ef571919da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-723f9934-9c14-4881-befa-61ef571919da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:13.697533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:13.698607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:13.699439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 371/734] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:13.700657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.700657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.700657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.718122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:13.719293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:13.756443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6672a3-fb9f-4237-9a25-45fc06354cd3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.772744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.794292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 372/735] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:13.794292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.794498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.794498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.794498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.794498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.794858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:13.794858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.794858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.817145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 364/736] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:13.817950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.892380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e246-7d52-4ea8-8c5a-9f3f8e8a08ae None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.906494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:13.920788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13eea2-ba1a-414e-99d9-28e4eb00e91c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:13.925662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.926483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.926767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.927042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.936907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.938452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.953887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:13.954679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.954962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.955236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.955974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:13.956284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d5f27749-ad4e-4bb8-af4d-dca435599173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:13.956422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:13.957429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:13.963255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:13.963255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:13.963255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:13.963255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:13.963490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 373/737] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda85503-a2d3-4a7d-9a82-72d8c5a1c3d4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda85503-a2d3-4a7d-9a82-72d8c5a1c3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda85503-a2d3-4a7d-9a82-72d8c5a1c3d4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 374/738] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:13.963566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda85503-a2d3-4a7d-9a82-72d8c5a1c3d4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:13.968819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 365/739] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:13.969633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.008967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.062621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda85503-a2d3-4a7d-9a82-72d8c5a1c3d4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.067517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441c42f-8898-4d27-9bd3-35b2c98fab82 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.096039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.096701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.096979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.097268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.097968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.099358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.101109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.102193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.102478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.102752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.167587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.169044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ea5c1-68d1-41d9-921e-512ec508a71b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.201994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 375/740] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:13 2022] POST /identity/v3/services => generated 240 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:14.202839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.206482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.253754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.281581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 366/741] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 20:05:14.281581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.281581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.281581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.298397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:14.303961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:14.355661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a977-df56-4904-8ad3-0ee0987689d0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.386040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 367/742] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 20:05:14.386040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.396815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.397497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.397778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.398277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.399641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:14.399641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.400957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.412144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 376/743] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:14.413282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.470792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.501360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.509302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fedc14-616e-4c82-8fbe-1d78c259317b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.519769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740c05-e0c0-4a42-b7dd-4dd7ddfef931 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.543348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.544669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.544952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.545223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.556162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.556775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.557562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.557684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.557844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.558137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.568891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.570353 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:14.575618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:14.576503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:14.577395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:14.577395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:14.577395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:14.577395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:14.577395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 368/744] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:14.578153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:14.590867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:14.591826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:14.592723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:14.592723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:14.592723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:14.592723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:14.592723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 377/745] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:14.593006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.688376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae6250-8369-4037-a8f1-ff93892a3d42 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.700738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e6e00-9ec0-44ac-acb9-91fa73560c0c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.709755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.727010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.727680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.727962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.728234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.729007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.730469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.736808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.737604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.737886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.738193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.740312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:14.745992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.748084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:14.762385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:14.763605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 378/746] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:14.764565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.812855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 369/747] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:14 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:14.814275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:14.882073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea711e0-8493-4c11-a6b1-0c57c61339dc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.943537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:14.944714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:14.945147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:14.945838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:14.947249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:14.947989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:14.949585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:14.950652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3754fa3-8d67-4f00-b143-804629312567 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:14.964386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 379/748] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:14.965431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.004315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.005493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.005929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.006402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.007555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.008045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.010906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.024770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 370/749] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.024998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.032602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.078454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.099189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99d37d-aec8-4f21-b252-ab98d1be46b6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.143378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2935e5-a3f4-43ef-a276-47225c37e6a4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.149552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.150314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.150599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.150929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.151775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.152086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.154558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.178107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 380/750] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.179288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.188994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.190197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.190696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.191227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.199242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.200941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:15.218032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:15.219253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:15.220459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:15.224565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 371/751] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:15.224565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.313090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59636912-25c6-42ab-b2e3-00f940556feb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.323079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.335197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.338532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4a930-8e70-48e5-9426-b2f953367947 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.353892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.354959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.355389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.355836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.363605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.365245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.375838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.376658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.376944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.377536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:15.381589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:15.382722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:15.383755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:15.390044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.390182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 381/752] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:15.391512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.392114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:15.413373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:15.414352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:15.415254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:15.415254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:15.415254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:15.415254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:15.418104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 372/753] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:15.419004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.503610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa8c2-9bee-4626-8d76-44237ca8b0c0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.529377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04462971-ae21-4424-ad4f-c6df8e92f436 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.541155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.542143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.542648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.543118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.544342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.544868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.546574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.564927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.565691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.565905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 382/754] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.566006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.566263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.567181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.567499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.567612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.568933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 373/755] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.586984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.599614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.604390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.682609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd4a7-9969-4e3d-89a5-757513687f59 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.690424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec745e-abd0-4f48-9a87-9d6690e5e62c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.720178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.721228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.721768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.722382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.723308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.723834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.725731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.727367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.728132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.728421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.728695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.739715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 383/756] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.740895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.806683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.808320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.823131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.834700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:15.850338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e830e-6002-4cc6-b070-73536458fd55 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:15.887377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:15.889025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:15.889608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:15.890138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:15.891275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=f42e373971c649ea89f8764d6cf142dc) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:15.891790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:15.893624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:15.906459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 384/757] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/role_assignments?user.id=f42e373971c649ea89f8764d6cf142dc&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:15.918213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:15.918374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 374/758] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:15.920873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.058460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d75922-a6d9-4aa1-ac72-35d28cd2e081 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.074364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:16.098332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.099555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.099991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.100416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.114362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.116108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:16.138618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:16.139750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:16.140835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:16.140835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:16.140835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:16.140835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:16.142920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 375/759] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:16.154054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.154260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7954a5c-ade6-4ef0-8ec3-c1771049374a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.204813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.205612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.205891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.206191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.214267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.215175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:16.230040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:16.231008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:16.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 385/760] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:16.237260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.250130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:16.343208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603217-5487-4e09-ba25-749a2d2397a2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.374088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a5083-3c14-42ac-a898-2e69269b8674 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.384678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.387412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.387902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.389426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.390409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:16.391517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.393108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.417417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 386/761] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:16.427527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.443381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.444266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.444688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.445128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.445993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:16.446476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.449410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.472653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 376/762] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:16.478292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.521544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:16.547337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:16.586929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c7926-7589-4fff-b43f-32fdbb51b81f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.619902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072fc38-28ea-4829-a0ee-123750ea4044 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.632396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.633303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.633588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.633867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.636597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=4cf871584be84949a82a9da23fb16153, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.638908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6238a-a54e-4618-8973-9f55a43b7499 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.659993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.661102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.661581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.662085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.669851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.671576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-df138740-9709-4669-92b0-d8af53e55b69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:16.685920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:16.686914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:16.687846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:16.690097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 377/763] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:16.691532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17bbe-3fc3-4ae1-bf2f-b1461b118828 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.692094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17bbe-3fc3-4ae1-bf2f-b1461b118828 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.692565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17bbe-3fc3-4ae1-bf2f-b1461b118828 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.694362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 378/764] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:16.695246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17bbe-3fc3-4ae1-bf2f-b1461b118828 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.742753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 387/765] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:16 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:05:16.744562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.744862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.745147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.784800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:16.788723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:16.844194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:16.869178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17bbe-3fc3-4ae1-bf2f-b1461b118828 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:16.917370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 388/766] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:16.923232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:16.924196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:16.924684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:16.925194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:16.926444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:16.926954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:16.928566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:16.940460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:16.951356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 379/767] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:16.957771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.036302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:17.093965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5462f7-d7cb-4fde-88ef-c1d581d26e04 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.099016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c5aa8-2693-4c58-9c4b-ac4499430f84 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.136011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.136771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.137058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.137334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.138755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=1cc42255281f442bb146fe386f432cc2) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:17.139066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.140687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.144988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.145792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.146117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.146396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] PATH_INFO: `/v3/services/network` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.146841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 380/768] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/role_assignments?user.id=1cc42255281f442bb146fe386f432cc2&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:17.147072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.148439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-88a6fcf2-5240-48f9-af32-f35e9d09fb15 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:17.173009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:17.174082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:17.174957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.177794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 389/769] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:16 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:17.184014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.202436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:17.293915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e7fb-9076-4cb8-bfb9-5c53243f9c05 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.294362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e7fb-9076-4cb8-bfb9-5c53243f9c05 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.294529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e7fb-9076-4cb8-bfb9-5c53243f9c05 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.296701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 390/770] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:17.302937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.303239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.303496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.346326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:17.348652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:17.361236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58f29-5af8-4d87-a268-617a16eca909 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.410338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.412480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.412762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.413034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.413750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] RBAC: Adding query filter params (name=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:17.415311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.416683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.435630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 381/771] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:17.436432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.462499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 391/772] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:17.529260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.568602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:17.623770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b05f7e-6d51-4967-ab4f-050d16b96f28 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.634355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:17.689647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.691239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.691522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.691795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.692511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] RBAC: Adding query filter params (type=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:17.692817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.694384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.705328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 382/773] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:17.711747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.720941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e91ae2-4a2c-4977-abd7-49fe1a048349 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.789414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.790308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.790590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.790860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.791519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.792897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:17.834912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:17.835921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:17.836766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 392/774] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:17.855047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:17.864475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f42e-ffce-48c1-aa1e-361ec2bea357 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:17.939834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] RBAC: Adding query filter params (interface=public, service_id=d80d0610cd47423691d12a007bb2da8d, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:17.946054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:17.956535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:17.960874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 383/775] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/endpoints?service_id=d80d0610cd47423691d12a007bb2da8d&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:17.967174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:18.147292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4914b0-0e79-42c7-9a1c-4994e7e50326 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:18.148706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0011d1-dbc4-4f4a-9c1a-ffea2116e10c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:18.186438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:18.222736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:18.225578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.226915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:18.227409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.227847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.230200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.230612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:18.232661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.233089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.234312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:18.235515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:18.235991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:18.239238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:18.259546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 384/776] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:18.259983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 393/777] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:18.266411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:18.382311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:18.458672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55e9bb-aa05-4482-9e37-2def75adc449 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:18.523506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:18.527049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.527509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.527954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.529560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:18.542045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:18.544133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e4454-d79b-400f-9ef5-0ca88081e94d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:18.550585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 385/778] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:18.556659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:18.708799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d4971-05df-4f37-8f29-7203f57173d4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:18.714664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043ebc-728f-4fc6-986b-331695cb2c7c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.715250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043ebc-728f-4fc6-986b-331695cb2c7c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.715748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043ebc-728f-4fc6-986b-331695cb2c7c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.722999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 386/779] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:18.724523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.725046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.725541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.748066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:18.748671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:18.749344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.749632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.749911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.750570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:18.750753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] RBAC: Adding query filter params (interface=public, service_id=72d2bc36aebe40b6b650ed6f6cae3da9, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:18.751751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:18.753257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:18.761757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 394/780] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:18 2022] GET /identity/v3/endpoints?service_id=72d2bc36aebe40b6b650ed6f6cae3da9&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:18.769712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:18.830819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:18.836200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 387/781] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:18.859563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:18.889845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842030f-2b0f-43ed-a2ea-0ba96e1ba0e4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:18.943094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:18.943955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:18.944767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:18.945156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:18.946320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:18.949177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:18.950909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:18.958153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 395/782] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:18 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:19.049127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688de71-1f1a-4613-9753-700c594e3853 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:19.104472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:19.105211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.105492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.105765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.129450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:19.137311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:19.139011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:19.140018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 388/783] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:19.145246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:19.281312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a280866-dddc-4a5b-b7bb-a8c16137ff14 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:19.306388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:19.317722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:19.318562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.318905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.320569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.320569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:19.320569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:19.322172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:19.332049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 396/784] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:19.337608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:19.456685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471522b-8a3c-416f-aeec-6be76a1d1b79 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:19.478830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:19.501945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:19.503174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.503707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.504163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.527242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:19.529102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0fc9a0b7-ae7a-4c56-a5ec-cd33025739e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:19.552336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:19.553289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:19.554229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:19.554229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:19.554229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:19.554229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:19.556331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 389/785] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:19.562226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:19.570713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c4679-f17a-49e8-be29-4aa182e50030 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.571170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c4679-f17a-49e8-be29-4aa182e50030 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.571616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c4679-f17a-49e8-be29-4aa182e50030 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.573345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 390/786] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:19.578637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9547b9-9add-48c9-83c7-f94d589d69ee None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.579211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9547b9-9add-48c9-83c7-f94d589d69ee None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.579684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9547b9-9add-48c9-83c7-f94d589d69ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.606599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-de9547b9-9add-48c9-83c7-f94d589d69ee None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:19.611522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-de9547b9-9add-48c9-83c7-f94d589d69ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:19.742125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 391/787] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:19.742125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e5c0-730f-4670-82ee-803886e47d75 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.742125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e5c0-730f-4670-82ee-803886e47d75 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.742125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e5c0-730f-4670-82ee-803886e47d75 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.745501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 392/788] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:19.746708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.747016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.747267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.748254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc38781-9be1-49b7-af1d-3630415dee42 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:19.766435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:19.767676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:19.809550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:19.810296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:19.810583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:19.810859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:19.811912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:19.812222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:19.813596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:19.829444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 397/789] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:19.830821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:19.854898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 393/790] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:19.858134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:19.885978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:19.906006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:19.959813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42044f-0d85-49ac-833f-5df4bfc4e5ef None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:19.988059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f16371-8cd0-4a23-8ef7-496971a8b2f6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.004467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.005449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.005970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.006277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.020785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:20.024110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.025023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.025308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.025580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:20.029477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.030430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:20.031268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 398/791] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.032021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.037798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.039354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:20.055737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.057885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 394/792] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.060735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.161338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.163927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.174083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adb0e7-d59e-4bed-89db-1a7df904ac13 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.174326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aed8e-656f-49e1-9f48-7168fe43f1ee None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.210483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.211546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.212031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.212484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.213627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:20.214156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.214261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.215328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.215610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.216397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.216940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.227207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 395/793] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:20.228343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.238041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:20.240699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.241583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:20.242472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:20.246313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 399/794] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.247147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.337817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b4ea4-19c7-497c-a799-f9937c9f3efb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.347062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3281d7-cea6-4488-a6a5-16b0e2a6547f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.373450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.374297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.374674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.375049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.375982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:20.376408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.377912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.382128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.382921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.383327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.383606 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.395206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.396771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.396893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 396/795] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:20.398103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.410700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:20.415189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.416067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.417043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:20.417043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:20.417043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.417043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:20.421794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 400/796] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.423817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.442395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.507762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e10d-9fa1-4f76-bcaf-82eeb2933f90 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.529073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98500d59-77ac-4ce8-bf8d-2be943d50cf0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.546162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.547486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.547948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.548404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.549328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:20.550356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.552421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.564226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.565138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.565552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 397/797] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:20.566647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.569885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.570282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.583379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.585367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:20.609524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.610479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.611389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:20.611389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:20.611389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.611389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:20.614108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 401/798] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.615120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.666451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.678554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038ae27-73a9-4057-93a4-6ce1c1aa1b99 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.685776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.725039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.725835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.726459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.727040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.727824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:20.728385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.730420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.740948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee11e79-d761-4a45-97f6-bb341b06a7fd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.749983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 398/799] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:20.751870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.780057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.781193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.781671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.782164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.792649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.794421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:20.812604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:20.813493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:20.814414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:20.814414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:20.814414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:20.814414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:20.821110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 402/800] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:20.822185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.874519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c38cf-c97e-4478-bd65-d715b4fbfb56 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.914808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.921769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.922485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.923202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.923496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.924226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:20.924701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.926732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:20.943408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 399/801] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:20.944250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:20.944360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cea42-b8f8-49d4-af54-743a56b4bcf9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:20.956184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:20.984108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:20.985422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:20.985998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:20.986500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:20.994727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:20.996677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:21.014054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:21.015018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:21.015960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:21.018656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 403/802] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:21.019639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.071648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218c87a-76f1-4893-93e7-f367f1507f45 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.106918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.107591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.107874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.108152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.108877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:21.109185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.110579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.126124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 400/803] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:21.126124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.131230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef82e4-9c0d-4ab8-86a2-997994dd8e77 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.166471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.167274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.167557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.167832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.175024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.176325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.178404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-64a0f480-201b-4b6c-b873-2091a37197e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:21.192206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:21.193129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:21.193998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 404/804] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:21.195421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67fac-fe5e-4946-b14d-cf8dce6efcec None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.195421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67fac-fe5e-4946-b14d-cf8dce6efcec None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.195421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67fac-fe5e-4946-b14d-cf8dce6efcec None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.198292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 405/805] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:21.199518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b67fac-fe5e-4946-b14d-cf8dce6efcec None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.233884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.238096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ed5c0-166c-4dfa-a2de-9afa0354b07e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.275913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.276589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.276870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.277148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.277977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:21.278343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.279758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.301845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 401/806] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:21.301845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.325545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b67fac-fe5e-4946-b14d-cf8dce6efcec None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.361222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.362119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.362460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.362804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.370181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.371685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b159de1d-e973-4969-b6c9-8635073e5648 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:21.386886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:21.388058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:21.389032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 406/807] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:21.389622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.389909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.390184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.408065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:21.409301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:21.420630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2b7af-8bf7-4e78-9308-0f97dd3e1bc0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.433076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.458976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.459793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.460175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.460575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.461495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:21.461933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.463464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.476087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 402/808] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:21.482188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.486058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 407/809] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:21.486058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.513919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.592766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516cfc-3aea-4eb5-9319-f9094124c883 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.597109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd713e04-ff47-417b-92bd-ec478ef82bf9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.630994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.631697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.631803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.631980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.632252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.632422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.632673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.632946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.633098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:21.633400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.634876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.651588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 408/810] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:21.652212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.671339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.698922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.700366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.730993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.753465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e294e-5e3c-42b4-a6b8-4c94d7fd3e8c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.787259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.788131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.788412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.788683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/f42e373971c649ea89f8764d6cf142dc/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.797667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 403/811] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:21.798515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.803113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=f42e373971c649ea89f8764d6cf142dc, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.804559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82201375-459c-4f83-977b-54b60a063dbc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.865622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:21.894032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e5306-3dda-4e86-936e-6a52eb920712 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:21.911356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 409/812] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:21 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/f42e373971c649ea89f8764d6cf142dc/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:05:21.911356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd709-ba97-407c-90bc-018b58a9276c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.911657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd709-ba97-407c-90bc-018b58a9276c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.911859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd709-ba97-407c-90bc-018b58a9276c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.913317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 410/813] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:21.914036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cd709-ba97-407c-90bc-018b58a9276c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:21.933465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:21.934319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:21.934667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:21.934945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:21.935614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:21.937010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:21.949903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:21.951035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:21.951876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 404/814] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:21.952678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.045010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cd709-ba97-407c-90bc-018b58a9276c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.068451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee8cf-a554-4dc0-98f0-e6100272f1a7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.074631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.084097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.084898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.085178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.096252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.102879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.104257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.111311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.112105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.112383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.112654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.114676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-96fd2a7c-13e9-479f-a5f8-b48fb66352ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:22.122360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:22.126873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 411/815] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.127781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.131389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.134179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.144375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:22.145529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:22.153247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:22.154750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:22.155683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:22.155683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:22.155683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:22.155683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:22.156807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 405/816] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:22.158244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.230115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 412/817] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:22.230115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.277495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037e9-7c75-4b40-b1bb-79fbe7e8af63 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.314596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.315263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.315547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.315826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.316530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] RBAC: Adding query filter params (name=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:22.316838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.318219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.323288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 406/818] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:22.323895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.335142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3688d41-c379-403b-9a50-e90d62ab2fbf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.346613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.373557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.374256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.399877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.399877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.399877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:22.399877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.400322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.400402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.400402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 413/819] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:22.400402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.432310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478550-f5b2-426a-a13b-fd257a217265 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.469742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.470440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.470723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.470996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.471718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:22.472023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.473393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.485426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 407/820] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:22.486181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.502885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196d12-c258-4d65-8d74-2ef2fd311dda None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.538876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.539682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.539986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.540226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.540872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.542272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:22.554418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:22.555374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 20:05:22.556235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:22.561122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 414/821] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:22.561722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.593684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.594345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e4505-d249-4f7b-a641-3cebea02a581 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.607304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.630958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.631746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.632039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.632317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.633033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] RBAC: Adding query filter params (type=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:22.633341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.634768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.640216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 408/822] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:22.641002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.672234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4cb15-4c15-4075-a661-490d0344ff6b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.709140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.709942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.710257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.710537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.718513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.719973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:22.735695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:22.736657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:22.737547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 415/823] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:22.738050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.752975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba7f84-0558-466f-bf36-88f5dbcc350b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.789792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.790609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.790891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.791163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.799448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.800881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:22.815301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:22.816350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:22.817414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:22.818343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 409/824] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:22.819704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.834363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.843684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfba05-4aa7-47bd-8dfa-0c58b1463988 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.854353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:22.879817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.880613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.880895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.881166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.881865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:22.882220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.883789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.892776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 416/825] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:22.892776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:22.925551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a7aa1-8153-4ecb-b8bc-00cd58a6ae9b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:22.959416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:22.960159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:22.960503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:22.960842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:22.961643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:22.963100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:22.993611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92f482-f209-41ef-a795-0878d1e6d6a0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.027904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.028724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.029009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.029279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.030102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:23.030411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.031855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.044424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 417/826] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:23.045272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:23.065209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:23.079972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 410/827] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:23.079972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:23.171921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f927f-e104-4fc7-a756-01581d1cbd1f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.201152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:23.230426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.231310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.232302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.232662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.233625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:23.234165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.236160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.238597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc54-fac2-4487-a37a-91c289887098 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.255650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 418/828] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:23.255650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:23.297922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.299426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.299708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.299982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.300702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:23.301010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.302645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.313755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 411/829] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 20:05:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:23.315285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:23.346823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:23.382675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159b90-e051-4eae-a662-bc3b302330b7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.418673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.420117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.420397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.420683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/1cc42255281f442bb146fe386f432cc2/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.422724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=1cc42255281f442bb146fe386f432cc2, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.424415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c33830-1bf4-4daf-9511-a052cd94e595 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.431865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0d46b-d756-44e7-b827-32637bd5b75c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.465084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.466303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.466589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.466864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.467689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:23.468122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.469535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.475402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 412/830] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:23 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:23.475976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:23.493999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:23.544659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 419/831] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:23 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/1cc42255281f442bb146fe386f432cc2/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:05:23.603263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9caab-1db2-4e1a-9573-520258b270ec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:23.645374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:23.646377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:23.646854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:23.647278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:23.648215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:23.649813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305bd9a-0212-4ea3-a132-3a7c9a9cca5d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:23.729744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 413/832] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:24.149814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde01f8-b4a1-466d-999f-e4b06c993d81 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.150358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde01f8-b4a1-466d-999f-e4b06c993d81 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.150438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde01f8-b4a1-466d-999f-e4b06c993d81 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.151928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 420/833] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:24.157369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.157664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.158100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.175122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:24.176313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:24.248958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:24.259727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 414/834] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:24.282940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde01f8-b4a1-466d-999f-e4b06c993d81 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:24.394649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde01f8-b4a1-466d-999f-e4b06c993d81 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:24.425924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:24.431574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:24.432531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.432915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.433308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.444603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:24.452659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:24.453599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:05:24.454462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:24.458135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 421/835] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:05:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:24.460147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:24.563484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61817da8-9e32-454c-81fb-773f061b3659 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:24.598265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:24.598939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.599224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.599504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.600321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:24.600632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:24.602315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:24.615532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 415/836] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:05:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:24.616906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:24.723756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33636-4b1a-4228-8974-2725a143da48 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:24.758734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:24.760631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.760631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.760631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.771368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:24.773030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:24.791148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:24.792117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:24.793009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:24.793009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:24.793009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:24.793009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:24.795668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 422/837] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:24.801322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:24.863465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:24.909044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f70676-1107-460b-b998-d8175b9fe0c3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:24.944690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:24.945461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:24.945811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:24.946138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:24.946872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:24.947185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:24.948552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:24.965390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 416/838] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:24.967248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.017944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:25.073291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1476-82ca-46f9-8aa4-996deef3917e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:25.108479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:25.109564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.110049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.110534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.118212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:25.119855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:25.134651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:25.135628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:25.136598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:25.143836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 423/839] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:25.145344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.250062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24a16-828b-4a72-8c7b-64638b3f63ed None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:25.268138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:25.284936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:25.285621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.285906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.286216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.287059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:25.287372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:25.288800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:25.312433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 417/840] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:25.313754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.418325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187aa6f7-ecdf-4891-b338-6c5c73c83b1d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:25.421421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:25.457040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:25.458210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.458748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.459227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.533582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:25.534990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a0100-09ab-4a01-943d-073b84e0701a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:25.641325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 424/841] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:25.647294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.752182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f1014-140a-47fb-8e7f-dba2e153a3d5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:25.788415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:25.789568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.790083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.790563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.801431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:25.803394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:25.808101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287f6d5-7c96-428f-bf25-2d113f4d7e9d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.808400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287f6d5-7c96-428f-bf25-2d113f4d7e9d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.808661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287f6d5-7c96-428f-bf25-2d113f4d7e9d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.810389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 425/842] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:25.816341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.816661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.816896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.823716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:25.824790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:25.826378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:25.827284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:25.827284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:25.827284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:25.827284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:25.831356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 418/843] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:25.832791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.842454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:25.844062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:25.883983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:25.925366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 426/844] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:25.940579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004f7e8-f732-4dfb-8675-6ce6bf9b379d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:25.941366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:25.976768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:25.977688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:25.978109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:25.978493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:25.979337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:25.979749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:25.981366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:25.998817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 419/845] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.000607 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.048211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e00543-287f-479e-9641-231b2348b9f1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.081483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.082300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.082688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.082968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.084780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.091938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.093771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b015d953-b588-4827-94e6-b27e1fe0b409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:26.101356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:26.102627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:26.103580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 427/846] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:26.104272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafe185-a4e1-4e09-b1f2-8b512614a300 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.104352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba96075-18ca-48a2-b51e-96a802d75e3b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.104763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba96075-18ca-48a2-b51e-96a802d75e3b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.104763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba96075-18ca-48a2-b51e-96a802d75e3b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.106471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 428/847] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:26.107531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e809e-fd65-41ba-8b10-7d099ba3552e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.107810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e809e-fd65-41ba-8b10-7d099ba3552e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.108085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e809e-fd65-41ba-8b10-7d099ba3552e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.109555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 429/848] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:26.110776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e809e-fd65-41ba-8b10-7d099ba3552e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.141981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.142889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.143170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.143449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.149942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.151397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bc3f8ce9-d708-49ae-ae79-8f7b04673dc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:26.165663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:26.166625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:26.167518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 420/849] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:26.167999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf291e4-616f-44a5-91ba-835ef45abf06 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.168265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf291e4-616f-44a5-91ba-835ef45abf06 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.168562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf291e4-616f-44a5-91ba-835ef45abf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.185304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-baf291e4-616f-44a5-91ba-835ef45abf06 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:26.186500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-baf291e4-616f-44a5-91ba-835ef45abf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:26.217904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e809e-fd65-41ba-8b10-7d099ba3552e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.251451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.252116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.252400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.252685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.253419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.253727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.255144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.259911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 421/850] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:26.259911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.259911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.259911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.267802 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 430/851] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.268523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.279809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:26.280974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:26.333792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.345593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 422/852] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:26.346448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.356151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.378425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220def24-2526-4cb4-9c5f-bc9dfcfc9a12 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.411242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.411901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.412183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.412458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.413241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.413552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.415022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.426157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 431/853] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.426936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.448260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad4abc-fd29-44cc-a76b-62d6d9888a79 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.482431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.483246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.483528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.483804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.493460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.494909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:26.512441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:26.513478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 423/854] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:26.519164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.524332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911b96b-0f9c-4e1e-a334-87fc29a5219d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.540253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.557845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.558684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.558967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.559267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.560554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.561906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:26.574568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:26.575662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:26.576643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:26.577593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.580604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 432/855] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:26.581418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.619406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085976f6-b50e-4995-be2d-19c29cccbb8e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.651994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.652780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.653103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.653378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.654034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.655431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:26.666917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:26.668067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:26.669435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 424/856] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:26.670261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.681501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38364731-1fb5-4b84-aece-7e9eb5badb94 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.715014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.715718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.716002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.716278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.717101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=f42e373971c649ea89f8764d6cf142dc) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.717412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.718867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.738263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 433/857] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/role_assignments?user.id=f42e373971c649ea89f8764d6cf142dc&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.738263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.770591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b78a6c-6a98-4eb1-ab9f-9770060c8b4f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.776625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.795860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:26.811118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.812218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.813044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.813507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.815087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.815605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.817843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.832281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 425/858] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.833280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.844179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb731-08a7-4823-9609-cf31b3106976 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.881949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.882696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.882985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.883253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.883982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.884287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.885745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:26.891055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 434/859] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:26.891897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:26.953806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ccde-44b9-465f-a298-6ce825b9c5a6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:26.994588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:26.995262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:26.995553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:26.995828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:26.996511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:26.996859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:26.998249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.003022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 426/860] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:27.003738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:27.018924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ecb71-a14e-4f90-9740-942d03c001cb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.028748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:27.054867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.055666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.055948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.056222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.056871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:27.064360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.065826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:27.080795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:27.081856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:27.083175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 435/861] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:27.083772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:27.113352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec136e9-9bc9-4742-9739-492f1b4a6201 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.149121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.149716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.150005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.150304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.151007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:27.151314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.152918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.158557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 427/862] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:27.159416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:27.200227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb53c6-7542-4710-89e9-eb554b4b24ab None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.252590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.254364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.254654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.254928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.255671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:27.255979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.257395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.264290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 436/863] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:27.298004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d302f7-2acb-468a-a688-c4ceedbe1d6e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.314430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:27.341239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.342421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.342882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.343329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.344413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:27.344878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.346591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.368192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 428/864] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:27.374379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:27.417541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:27.493152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec72a1-de88-4b02-981d-eef72ea9c14b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.529568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.530251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.530541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.530818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.602404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.603868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.634962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:27.708014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 437/865] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:27.713459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:27.820186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad65e7-b9fc-46bf-874b-144a57b12ca8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:27.853836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:27.854668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:27.854946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:27.855220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:27.865199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:27.866672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:27.884901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:27.891990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:27.892887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:27.892887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:27.892887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:27.892887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:27.892887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 429/866] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:27.893178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:28.000995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5dfa7-5e43-4447-bbcd-fd64f1b04e9e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:28.031521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:28.036679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:28.037345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:28.037623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:28.037895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:28.038647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:28.038950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:28.040317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:28.053420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 438/867] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:28.059032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:28.088294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:28.169500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c04fe-c407-4afd-9139-be8ff28e16ef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:28.204330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:28.205133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:28.205417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:28.205691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:28.212484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:28.213933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:28.232753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:28.233641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:28.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 430/868] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:28.239645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:28.348044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d11877-3610-4414-a0f8-5d0cced7cdc7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:28.384157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:28.384883 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:28.385082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:28.385349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:28.386154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:28.386459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:28.387975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d547562-2845-430e-bb1b-e5696e955651 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:28.400010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 439/869] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:28.406363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:28.497073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:28.515601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f48940-dc36-4af4-bf86-687ea3ac4a7c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:28.551589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:28.552254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:28.552540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:28.552817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:28.553652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=1cc42255281f442bb146fe386f432cc2) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:28.553966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:28.555416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd53d4-153c-45cd-82af-38544cf4b0e4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:28.562039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 431/870] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:28 2022] GET /identity/v3/role_assignments?user.id=1cc42255281f442bb146fe386f432cc2&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:29.137887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b007b-ad38-4a55-89e1-f3b393c8fa43 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.138322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b007b-ad38-4a55-89e1-f3b393c8fa43 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.138510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b007b-ad38-4a55-89e1-f3b393c8fa43 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.140107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 440/871] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:29.146248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.146248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.146248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.164229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:29.165703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:29.237478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 432/872] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:29.264184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b007b-ad38-4a55-89e1-f3b393c8fa43 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.321817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:29.372332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b007b-ad38-4a55-89e1-f3b393c8fa43 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:29.407679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:29.408414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.408701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.408975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.419900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6a0d6ab9-d35d-4700-837c-fbc36e1dc740 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:29.428370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:29.429419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:29.430280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:29.434694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 441/873] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:29.436821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.450963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f206a-cedc-4208-a2f9-df14aa989b30 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.451296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f206a-cedc-4208-a2f9-df14aa989b30 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.451520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f206a-cedc-4208-a2f9-df14aa989b30 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.453088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 442/874] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:29.458886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.459393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.459814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.476912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:29.478356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:29.540230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:29.566911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc90267-c4d5-4c30-bb1d-30b643fbd551 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:29.582036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 443/875] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:29.597685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.607553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:29.608536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.609013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.609478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.610468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:29.610970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:29.612816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:29.622452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 433/876] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:29.629647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.651539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:29.708319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ff57-8804-4042-970e-f82c8c98d131 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:29.741056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007982c-c447-4c27-94a9-5e5d6a993c42 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:29.744509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:29.745181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.745462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.745734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.746503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:29.748441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db43746-9991-4178-8e49-9b7c18540e0b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:29.775119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:29.775914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.776197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.776469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.783581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:29.787378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:29.788808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6df1f47f-deea-4a24-9acb-78a33e41cf4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:29.806317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:29.807220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:29.808166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:29.808166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:29.808166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:29.808166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:29.808166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 434/877] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:29.809053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a20b-8565-454f-bffc-38dbbc055603 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.809217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a20b-8565-454f-bffc-38dbbc055603 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.809933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a20b-8565-454f-bffc-38dbbc055603 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.811351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 435/878] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:29.815865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3a20b-8565-454f-bffc-38dbbc055603 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.818700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 444/879] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:29 2022] POST /identity/v3/services => generated 269 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:05:29.819913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c676b4-c19f-420e-9613-ef891278382b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:29.820501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c676b4-c19f-420e-9613-ef891278382b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:29.820778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c676b4-c19f-420e-9613-ef891278382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:29.841286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-83c676b4-c19f-420e-9613-ef891278382b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:29.842904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-83c676b4-c19f-420e-9613-ef891278382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:29.938095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 445/880] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:29 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:29.957957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3a20b-8565-454f-bffc-38dbbc055603 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:29.965152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c676b4-c19f-420e-9613-ef891278382b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:29.998210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:29.999182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.000131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.000622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.001097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.002126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:30.002645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.004252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.017196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 436/881] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:30.022864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.094484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:30.099977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c676b4-c19f-420e-9613-ef891278382b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:30.136760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.137467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.137749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.138047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.138785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.140153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.160050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829c19-a238-4eab-bb02-674501a9c112 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:30.194917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.195732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.196011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.196298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.204659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.206205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:30.224717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:30.225714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:30.226983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:30.229475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 437/882] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:30.230851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.242467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 446/883] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:29 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:05:30.354070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98517c4-de54-41d9-af31-3434d87d1e3d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:30.366662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:30.399562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.400253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.400531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.400801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.401645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:30.401953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.403618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.424251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 438/884] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:30.430789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.527013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:30.539715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ca37-353f-4e9d-ba8e-e432d52c0212 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:30.574514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.575344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.575721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.576092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.652226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.654300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b3b67-ed01-448a-8eb9-3d2daa8b7ff8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.745894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:30.763446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 447/885] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:30.764803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.846514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2124-cdc2-4be5-81d9-33f95f3140d7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.846709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2124-cdc2-4be5-81d9-33f95f3140d7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.846981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2124-cdc2-4be5-81d9-33f95f3140d7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.848343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 448/886] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:30.853472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.853755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.854042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.869298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7643791-5db1-41d8-a940-beca49c6cf82 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:30.872283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:30.873465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:30.915114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:30.916016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:30.916369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:30.916717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:30.931135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:30.932934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:30.956969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:30.958035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:30.959137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:30.959137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:30.959137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:30.959137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:30.959389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 449/887] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:30.962718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 439/888] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:30.964068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.977535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:30.998193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:31.088283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96b53-841e-4364-9df1-4818187c96bc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.102816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05011735-dd58-4c0e-8f33-7e12f186af93 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.113312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:31.143213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.143928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.144224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.144500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.145273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:31.145581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.147017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.148787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.150631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.151449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.152126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.153654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.158803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.165970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 450/889] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:31.167433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:31.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:31.179332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:31.180620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:31.189068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 440/890] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:31.190644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:31.290778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a630e-f5da-46a1-889c-2aaa4cf8b1e4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.306119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:31.324537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.325829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.326170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.326449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.332822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5461-9cd3-40a5-a0ad-daf8ac3ec951 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.333337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.334827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:31.350397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:31.351605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:31.352492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:31.356710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 451/891] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:31.357898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:31.375509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.376558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.377138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.377621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.378613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:31.379171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.380862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.391786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 441/892] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:31.392849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:31.399584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:31.462281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72340-6193-4d90-bdb9-12e0fad34d3a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.498326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.498994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.499275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.499551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.500333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:31.500642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.502097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.506578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f937f8-37d9-4ea5-ab6e-7d31b7a10eae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.519368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 452/893] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:31.520670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:31.543970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.545078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.545658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.546310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.547221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:31.547740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.549516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a0cb-6ac0-4925-a395-a81e50fd5416 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.562527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 442/894] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:31.630614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:31.635213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f88d-62db-438b-a0c1-a04ebf2842f9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:31.674776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:31.675446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:31.675730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:31.676007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:31.676886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] RBAC: Adding query filter params (role.id=fc9526fa70d14df89002a67b4f550d59, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=f42e373971c649ea89f8764d6cf142dc) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:31.677242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:31.678962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d276e-c35a-44ce-9f9e-dbec54a9c108 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:31.690159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 453/895] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:31 2022] GET /identity/v3/role_assignments?user.id=f42e373971c649ea89f8764d6cf142dc&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=fc9526fa70d14df89002a67b4f550d59 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:32.171879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ad62-197c-48a6-85d2-b9eae12e63ed None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.172272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ad62-197c-48a6-85d2-b9eae12e63ed None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.172454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98ad62-197c-48a6-85d2-b9eae12e63ed None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.174139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 443/896] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:32.179527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.179820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.180102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.196812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:32.198004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:32.275827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 454/897] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:32.299342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98ad62-197c-48a6-85d2-b9eae12e63ed None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.310357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:32.412485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98ad62-197c-48a6-85d2-b9eae12e63ed None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:32.448729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:32.449457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.449772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.450074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.450731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:32.452137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d43064eb-84f1-4937-8612-2e90323c52b9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:32.464463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:32.465966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.466857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 444/898] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:32.472681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.566374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:32.575683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df245a-2114-460b-8bbe-d699fb2eba63 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:32.582161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5259e-68ae-433c-aadb-bc5df25c3e15 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.582455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5259e-68ae-433c-aadb-bc5df25c3e15 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.586871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5259e-68ae-433c-aadb-bc5df25c3e15 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.586871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 445/899] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:32.589187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.589472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.589745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.607015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:32.608187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:32.611707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:32.612433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.612717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.612992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.613699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:32.614006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:32.615393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:32.620500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 455/900] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:32.626999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.686178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 446/901] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:32.706901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.709270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:32.744047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d16494-e4c4-4e91-bc68-9584ba244e34 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:32.781855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:32.782561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.782844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.783114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.783919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:32.784226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:32.785713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:32.791122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 456/902] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:32.798074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.834424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:32.848054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc7ba8-5421-40e2-91d3-6109fe7264e7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:32.896091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:32.897395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.897692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.897983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.898821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:32.900523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:32.910117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a762d-e78c-438c-b725-07d737a8f0c3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:32.916216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:32.919587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:32.920454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 447/903] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:32.924104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:32.955637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:32.956388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:32.956736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:32.957399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:32.957961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] RBAC: Adding query filter params (interface=public, service_id=da8f02e2fa1842438b6b0d54c130bcb3, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:32.958393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:32.959877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:32.969691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 457/904] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/endpoints?service_id=da8f02e2fa1842438b6b0d54c130bcb3&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:32.970991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:33.015314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:33.059928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87758d-2cbf-4a11-a2c2-e8a724b7dfd4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:33.078166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7c67-3439-4e82-a6ca-f95290e5d599 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:33.094351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:33.102924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:33.103831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.104191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.104853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.105950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:33.106352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:33.108314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:33.114474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 448/905] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:33.119672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:33.119814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:33.120485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.120833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.121185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.122497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:33.124128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:33.134099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 458/906] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:32 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:05:33.240900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8219be-2495-4693-a6bc-1063d500a82e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:33.278031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:33.278738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.279176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:33.279667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.279959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.280696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:33.281012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:33.282576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:33.289356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 449/907] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:33.295715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:33.408652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc629b-e353-4f91-b331-38feb9d12dcb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:33.446089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:33.446804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.447089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.447363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.448156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] RBAC: Adding query filter params (interface=public, service_id=f3dd4bca057746c1b2dac2082694af6a, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:33.448467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:33.449871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77fae7-d471-4220-873a-f7af0761c852 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:33.455460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 459/908] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:33 2022] GET /identity/v3/endpoints?service_id=f3dd4bca057746c1b2dac2082694af6a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:33.463128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:33.570606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2773-2180-4c9c-b2a4-2bac379054ee None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:33.607331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:33.608418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.608881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.609334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.610242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:33.611795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ff0c3-f49f-42c6-abf1-d4550c913b7b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:33.620413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 450/909] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:33 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:05:33.876618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a08fc-5c99-4405-9b6b-c9e1c375e3f7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.877109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a08fc-5c99-4405-9b6b-c9e1c375e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.877213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a08fc-5c99-4405-9b6b-c9e1c375e3f7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.878790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 460/910] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:33.884189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636f03-4d6b-4853-87ff-dfcd677bac16 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.897355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636f03-4d6b-4853-87ff-dfcd677bac16 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.897355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636f03-4d6b-4853-87ff-dfcd677bac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.901811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-78636f03-4d6b-4853-87ff-dfcd677bac16 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:33.903583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-78636f03-4d6b-4853-87ff-dfcd677bac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:33.959799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:33.974791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 451/911] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:33.974978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1240-383e-4f4b-9acf-bbc93bb5c8fe None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.975156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1240-383e-4f4b-9acf-bbc93bb5c8fe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.975502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1240-383e-4f4b-9acf-bbc93bb5c8fe None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.977231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 461/912] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:33.983575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:33.983868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:33.984181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:33.997789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c1240-383e-4f4b-9acf-bbc93bb5c8fe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.006334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:34.007964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:34.014391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:34.080419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 452/913] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:34.103189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.127608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c1240-383e-4f4b-9acf-bbc93bb5c8fe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.167966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.169027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.169560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.169989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.170906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:34.172430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:34.207741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74576e2e-be43-40ad-a77c-0a5f11711235 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.230769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:34.246069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 462/914] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:33 2022] POST /identity/v3/services => generated 241 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:05:34.255684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.256980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.257503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.257994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.271343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:34.281323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:34.282663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:34.283681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:34.286401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 453/915] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:34.292992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.374102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:34.413996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396016f7-013b-400b-a8fd-b381407e48aa None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.451664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.452431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.452704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.452976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.453725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:34.454045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:34.455425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:34.466105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 463/916] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:34.470820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.577973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec764fa-f4a0-4e53-9c69-f41b22a7d4c8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.612826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.613978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.614498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.614984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.626394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:34.628078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:34.645832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:34.646752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:34.647647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:34.647647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:34.647647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:34.647647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:34.649860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 454/917] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:34.659084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.772541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbc83c-75db-44aa-a19e-9efeaca3d8b2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.809803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.810513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.810791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.811062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.811844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:34.812146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:34.813561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:34.815178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:34.827108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 464/918] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:34.832644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:34.864000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:34.942684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d141-e9ee-4573-af27-02c83829391d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:34.978282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:34.979410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:34.979892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:34.980338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:34.988384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:34.990053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:35.006106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:35.007068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:35.008013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:35.010264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 455/919] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:35.019109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.128807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bdab1-e795-492c-8ffa-9c311151e191 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.164558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.165510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.165933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.166386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.167360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:35.167809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.170042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.188088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 465/920] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:35.195157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.295737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.311414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abc178-bf4b-457d-9563-3073c2061f6c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.350313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.351124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.351658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.351932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.363463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.364926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-da3a4073-4aea-4a53-a2f0-2e749178da4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:35.382852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:35.383790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:35.385009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:35.385009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:35.385009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:35.385009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:35.385009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 456/921] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:35.395094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.451549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.500434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732f14-97ee-45f6-9692-74fc85f24668 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.518574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5664d2-9dcd-4d1e-8b3e-20aba09a80ee None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.519090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5664d2-9dcd-4d1e-8b3e-20aba09a80ee None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.519171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5664d2-9dcd-4d1e-8b3e-20aba09a80ee None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.520736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 457/922] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:35.526194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.526488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.526756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.538277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.538959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.539856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.540411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.541162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:35.541783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.543240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.545676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:35.547000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:35.557315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 466/923] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:35.562409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.630652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 458/924] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:35.649179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.680523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5cd7-99f1-4e63-8534-6f363c60bcb8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.694512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.718332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.719519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.720036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.720619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.721872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.731237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.733932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:35.752669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:35.753665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:35.754601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:35.758821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 467/925] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:35.760231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.772547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a75a2-d42f-4b83-9583-26c500a35227 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.817228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.818680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.819166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.819629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.820593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.822692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:35.838904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:35.840079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 20:05:35.841780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:35.844736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 459/926] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:35.851020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:35.882870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0dbda-3357-48c1-acec-bf8a7d0660ac None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.925534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:35.926146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.926433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.926711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.927493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:35.927859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:35.929339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed47a7-2b22-420c-9cbe-fa54634c5f36 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:35.942055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 468/927] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:35.942523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b5055-c3dc-4430-a668-867f2a56461c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.942808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b5055-c3dc-4430-a668-867f2a56461c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.943083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b5055-c3dc-4430-a668-867f2a56461c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.944427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 469/928] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:35.950224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:35.950590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:35.950874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:35.964646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2a39-fbcf-48a5-9531-169bc4392eb9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:35.968102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:35.969294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:35.993317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.997892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:35.999550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.006210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.008366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 460/929] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:36.014424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.050039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 470/930] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:36.050039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.136135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eca31-aa14-4a24-bfb7-5029c9e82cd5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.151438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aebb09-a49d-4795-9cfd-34498da3f508 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.172913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.174279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.174570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.174842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/f42e373971c649ea89f8764d6cf142dc/roles/fc9526fa70d14df89002a67b4f550d59` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.177344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=f42e373971c649ea89f8764d6cf142dc, role_id=fc9526fa70d14df89002a67b4f550d59)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.178749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.185785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.186490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.186789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.187050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.187760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:36.188070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.189444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.194297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 471/931] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:36.195064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.227381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:36.270314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 461/932] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:36 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/f42e373971c649ea89f8764d6cf142dc/roles/fc9526fa70d14df89002a67b4f550d59 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:05:36.270314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.329548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908abcf-76e4-43b8-b8cb-3cecdc49e7cc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.344225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:36.380965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.381850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.382266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.388285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.388285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.388285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.395782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6fdb4-48f3-485f-b31a-1d56360a39de None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:36.401844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:36.402984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 20:05:36.403836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.406186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 472/933] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:36.409536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.454503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.455103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.455382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.455655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.456408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.457774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.523580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:36.533342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8110c7-893b-47e3-80ab-cbd05b65bce8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.539379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 462/934] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:36.585008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.586335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.586827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.587303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.588222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:36.588732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.590908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ea463-5376-4901-b209-827ee0fa236c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.597639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 473/935] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:36.603303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.683055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95555120-df64-4374-b294-c3456e5297e0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.683426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95555120-df64-4374-b294-c3456e5297e0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.683620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95555120-df64-4374-b294-c3456e5297e0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.685882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 474/936] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:36.691061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827a83-90f5-4a99-b178-635741905042 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.691348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827a83-90f5-4a99-b178-635741905042 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.691611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827a83-90f5-4a99-b178-635741905042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.709508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8e827a83-90f5-4a99-b178-635741905042 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:36.710660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8e827a83-90f5-4a99-b178-635741905042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:36.730283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a26e37-e7b8-4921-8eeb-468e2fadea9a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.747867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:36.778463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.779221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.779429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.779704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.780442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:36.780795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.782215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.788510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 463/937] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:36.793865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 475/938] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:36.796885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.818660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827a83-90f5-4a99-b178-635741905042 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:36.906598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:36.912190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6378451-3056-40fd-bb26-9388c7181a5f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.927373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827a83-90f5-4a99-b178-635741905042 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:36.948860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.949675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.949935 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.950247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.950993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.952429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.965688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:36.966870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:36.967347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:36.967811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:36.968687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:36.970383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:36.983948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:36.984951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:36.985786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:36.992212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 476/939] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:36.992866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:37.070196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 464/940] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:37.108132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62ae8-5796-4b74-90ba-e386587bef52 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:37.155502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:37.156577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:37.157076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:37.157547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:37.158508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:37.159031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:37.160647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:37.170394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 477/941] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:37.176030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:37.212845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:37.291609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04f5d6-5444-49c0-8890-e4c60914b7e8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:37.327104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:37.328187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:37.328667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:37.329132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:37.330080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:37.330584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:37.332146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:37.337717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 465/942] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:37.344286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:37.354702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:37.455549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0d5cb-056e-4a03-9d0b-d4a0836af280 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:37.491193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:37.492213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:37.492694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:37.493181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:37.494469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] RBAC: Adding query filter params (interface=public, service_id=3f547e48c9d34461a2e2eb3c2e7e7868, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:37.494979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:37.497082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366cf8-f52d-4269-9cb4-e0c77efe48e8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:37.503429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 478/943] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:37 2022] GET /identity/v3/endpoints?service_id=3f547e48c9d34461a2e2eb3c2e7e7868&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:37.510047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:37.603319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:37.608987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266912d-28f4-4eaa-bdeb-44b1944641fc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:37.643471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:37.644140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:37.644421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:37.644696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:37.645374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:37.646754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ef796-caf0-4ce0-a981-12ef86be6464 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:37.653049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 466/944] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:37 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:05:38.531715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1992-1a91-462f-bdff-48e93f516267 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.532204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1992-1a91-462f-bdff-48e93f516267 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.532204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b1992-1a91-462f-bdff-48e93f516267 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.533800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 479/945] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:38.539208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.539501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.539785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.556370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:38.557566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:38.623769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 467/946] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:38.647449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b1992-1a91-462f-bdff-48e93f516267 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:38.718431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:38.757208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b1992-1a91-462f-bdff-48e93f516267 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:38.794598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:38.795347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.795629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.795904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.806933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-8a6ece5e-3ae3-438b-bbbf-0fb04a2d88b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:38.815312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:38.816235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:38.817050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 480/947] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:38.822898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:38.916278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:38.925998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafc55d-b798-482e-b865-9ef7b5edfd78 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:38.945482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd451-bedd-484f-8d2d-9931baa032d5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.945790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd451-bedd-484f-8d2d-9931baa032d5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.946057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fd451-bedd-484f-8d2d-9931baa032d5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.947689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 481/948] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:38.955516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.955992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.956374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.969015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:38.969714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:38.969999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:38.970313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:38.971377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:38.971690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:38.973062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:38.975926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:38.977589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:38.983253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 468/949] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:38.988944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.066413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 482/950] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:39.084099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.115274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86fc02-85a7-4f4c-ae12-48eb23a4adf6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.154372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.161768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.163278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.164362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.164862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.180902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.183588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.209344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e729e-dfda-45f4-9bfc-a09615baf0bc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:39.210079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:39.210970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:39.211864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:39.211864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:39.211864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:39.211864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.220012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 469/951] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:39.221160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.225701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.259214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.260322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.260725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.261095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.262442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.264076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:39.279031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:39.280221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:39.281076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 483/952] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:39.288168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.353725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec989e-ac2d-40da-86c0-2d36601522f6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.393579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.394418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.394706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.394982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.395706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:39.396223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.397381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f0e5-dc17-42e2-a21f-1dc5f4c52526 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.404928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3756a-10ab-41a3-a4e1-c9e56cc16285 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.414412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 470/953] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:39.414412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aada-86ef-468b-b962-1f7d5e65e60f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.414412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aada-86ef-468b-b962-1f7d5e65e60f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.414412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aada-86ef-468b-b962-1f7d5e65e60f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.414412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 471/954] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:39.418912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447aada-86ef-468b-b962-1f7d5e65e60f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.432813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.445678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.446729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.447013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.447299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.448002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:39.448308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.449781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fec51-1f44-413e-8f34-cafcf2856a92 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.454690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 484/955] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:39.456208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.456500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.456777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.472582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:39.473759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:39.492580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.533289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447aada-86ef-468b-b962-1f7d5e65e60f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.544994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 485/956] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:39.545651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.574258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.575115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.575394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.575857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.583499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.584919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:39.600015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:39.601072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 472/957] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:39.602357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.660293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be990c7a-0bf8-4a42-b510-950774aa54ef None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.678010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.695420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.696091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.696425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.696712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.697349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:39.697652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.699061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.714066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 486/958] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:39.714066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.727648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919ba9-5e42-4c87-9522-df46b67d5132 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.756704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:39.768895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.769702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.774175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.774175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.774175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.774175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.775148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING keystone.server.flask.common [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 20:05:39.826796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.sql.core [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 20:05:39.826796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 20:05:39.826796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [parameters: {'id': '47ebae57a1dc41ae800c48a1526a6fc3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 20:05:39.826796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47ebae57a1dc41ae800c48a1526a6fc3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47ebae57a1dc41ae800c48a1526a6fc3. May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:05:39.834620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 20:05:39.835556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application yield resource May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.rollback() May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.force_reraise() May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise self.value May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application session.commit() May 23 20:05:39.836645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._prepare_impl() May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.flush() May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._flush(objects) May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application compat.raise_( May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application flush_context.execute() May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 20:05:39.838161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rec.execute(self) May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 20:05:39.839447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:05:39.842084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [parameters: {'id': '47ebae57a1dc41ae800c48a1526a6fc3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.844372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:39.846327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47ebae57a1dc41ae800c48a1526a6fc3. May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 473/959] 10.222.0.16 () {60 vars in 1140 bytes} [Mon May 23 20:05:39 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:39.847481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.855103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43336b58-bab5-4508-a540-264ac5e0d5e8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.907425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:39.908368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.908826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.909869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.911214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:39.911680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:39.913581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742aa1f-2b87-4f4e-a01e-0784f200202d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:39.937529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 487/960] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:39.940784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da61b-428d-4387-b242-c4bcca85094e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:39.941071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da61b-428d-4387-b242-c4bcca85094e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:39.941346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da61b-428d-4387-b242-c4bcca85094e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:39.942972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 488/961] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:39.946930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da61b-428d-4387-b242-c4bcca85094e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:39.968041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e13719-8f36-445e-8206-ad551d3e876d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:39.995879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.004022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.004844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.005141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.005409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.012731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.014464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f5b33a78-a892-43f2-aacd-52170df72d73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:40.029150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:40.030098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 474/962] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.034378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.034988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.050708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:40.052328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:40.065313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da61b-428d-4387-b242-c4bcca85094e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.088891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.100480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.101594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.101879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.102179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.127529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 475/963] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:40.131197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.175523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.177252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.219973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.245629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18440bfe-a568-4a74-9579-77a4f3d3aa20 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.283541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.284209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.289104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.289237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 489/964] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:40.289367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.289471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.289559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:40.289559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.289559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.308752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 476/965] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:40.310992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.356926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.398065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966bd2f-b0a4-4d88-95b8-04fa9d1de159 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.435348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.436575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.437065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.437532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.438452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.440063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.441213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd651863-a987-4911-abe8-df03f9a2a8dd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:40.453916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:40.455053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:40.455991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:40.463848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 490/966] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:40.465237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.470773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.484884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.486733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.487223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.487713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.504312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.505766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:40.528884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:40.529860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:40.531835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:40.531835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:40.531835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:40.531835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:40.531835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 477/967] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:40.539594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.594830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb93e0-bc9e-4412-9108-f4d51e98798d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.636139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.644675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.650075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.654851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbaa39-d066-4c62-acdb-b90ac632dbde None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.656052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 491/968] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:40.661711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.700890 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.701893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.702490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.702778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.703761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:40.704106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.706174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.719950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 478/969] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:40.725630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.746299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.784840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce3d31-c21d-44ae-99c3-82e8ceeeaf20 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.831101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.831767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.832097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.832373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.833079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:40.833383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.835321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.846131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 492/970] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:40.847050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.847578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540abd3-8bd6-4833-b3c1-01e3992e4442 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:40.886628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:40.887719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:40.888181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:40.888636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:40.896102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:40.897837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:40.922044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:40.923009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 479/971] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:40.923928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:40.926261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:40.971590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda1752-d261-4802-9aba-6321352708ea None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:41.011991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:41.019346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:41.020009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:41.020690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:41.022833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:41.022833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:41.023631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:41.044150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27655c-3c9a-415c-87c2-dce30717842e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:41.079612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:41.080478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:41.080819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:41.081261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:41.082197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:41.082703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:41.084265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44becc70-b787-4bb7-81f8-b93507139723 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:41.087831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 493/972] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:41.102295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 480/973] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:41.108660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:41.234317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721abf0c-f4f1-44f0-b490-7655be444ffd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:41.272040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:41.272770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:41.273052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:41.273322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:41.274198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] RBAC: Adding query filter params (role.id=fc9526fa70d14df89002a67b4f550d59, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=f42e373971c649ea89f8764d6cf142dc) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:41.274509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:41.275964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c011e-cb9b-4292-a507-51f43c698f0f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:41.283564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 494/974] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:41 2022] GET /identity/v3/role_assignments?user.id=f42e373971c649ea89f8764d6cf142dc&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=fc9526fa70d14df89002a67b4f550d59 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:41.996292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca62b-b0e4-40cd-ae54-5f2093fc8e33 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:41.997152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca62b-b0e4-40cd-ae54-5f2093fc8e33 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:41.997583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca62b-b0e4-40cd-ae54-5f2093fc8e33 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:41.999401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 481/975] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:42.004940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cc024-87ff-4160-b88b-010e954f648a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:42.005789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cc024-87ff-4160-b88b-010e954f648a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:42.006312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cc024-87ff-4160-b88b-010e954f648a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:42.023487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-237cc024-87ff-4160-b88b-010e954f648a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:42.024981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-237cc024-87ff-4160-b88b-010e954f648a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:42.070855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:42.096218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 495/976] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:42.119920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dca62b-b0e4-40cd-ae54-5f2093fc8e33 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:42.184396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:42.229596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dca62b-b0e4-40cd-ae54-5f2093fc8e33 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:42.264652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769db180-cc86-473a-9605-9921fab2449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:42.265656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769db180-cc86-473a-9605-9921fab2449c None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:42.266182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769db180-cc86-473a-9605-9921fab2449c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:42.266655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769db180-cc86-473a-9605-9921fab2449c None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:42.267602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769db180-cc86-473a-9605-9921fab2449c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:42.269300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769db180-cc86-473a-9605-9921fab2449c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:42.340740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 482/977] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:42 2022] POST /identity/v3/services => generated 242 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:42.656389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0555e-e23e-4d1f-9b73-362ff6931bd8 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:42.656718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0555e-e23e-4d1f-9b73-362ff6931bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:42.656949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0555e-e23e-4d1f-9b73-362ff6931bd8 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:42.658604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 496/978] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:42.663883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:42.664172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:42.664448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:42.681698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:42.683039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:42.757973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 483/979] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:42.782608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0555e-e23e-4d1f-9b73-362ff6931bd8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:42.885252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0555e-e23e-4d1f-9b73-362ff6931bd8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:42.920815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:42.921412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:42.921767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:42.922120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:42.922842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:42.924408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:42.925745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING keystone.server.flask.common [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 20:05:42.967162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.sql.core [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 20:05:42.967162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 20:05:42.967162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [parameters: {'user_id': '2abecf89b9364c1297a4fc8e63be42df', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 20:05:42.967162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application conn.query(q) May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application result.read() May 23 20:05:42.984742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:05:42.985668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application next(self.gen) May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application yield resource May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application next(self.gen) May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self.session.rollback() May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self.force_reraise() May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise self.value May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application session.commit() May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 20:05:42.986652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._prepare_impl() May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self.session.flush() May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._flush(objects) May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application compat.raise_( May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application flush_context.execute() May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rec.execute(self) May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 20:05:42.987551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 20:05:42.988430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application result = self._query(query) May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application conn.query(q) May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application result.read() May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 20:05:42.989288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [parameters: {'user_id': '2abecf89b9364c1297a4fc8e63be42df', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:42.991271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 20:05:42.993236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:42.995377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 497/980] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 20:05:42 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:43.004201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.060175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:43.109756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77ad3-fdfa-4ab6-ae3d-f33c806c260a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:43.144214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:43.145026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.145307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.145579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.153749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:43.155315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2ba07575-b7d8-4fe4-b698-15cecdcd5ac8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:43.171939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:43.172905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:43.173851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 484/981] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:43.181160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.230254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:43.299028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ff58-b72d-48b5-bcdd-4fb1e674c80b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:43.337778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:43.338840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.339321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.339788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.341341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:43.341872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:43.343731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44641a88-5493-4f10-b425-b20260857af7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:43.382044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 498/982] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:43.481196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06f1f7-30cf-46c4-ac93-f245005170e6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.481842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06f1f7-30cf-46c4-ac93-f245005170e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.482445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06f1f7-30cf-46c4-ac93-f245005170e6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.484525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 485/983] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:43.491020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c39a5-6f53-44cc-957d-f66b3762b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.491678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c39a5-6f53-44cc-957d-f66b3762b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.492169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c39a5-6f53-44cc-957d-f66b3762b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.512992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d12c39a5-6f53-44cc-957d-f66b3762b2d7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:43.514949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d12c39a5-6f53-44cc-957d-f66b3762b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:43.591778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ff40-0037-4795-aef1-7329a2049d60 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.592086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ff40-0037-4795-aef1-7329a2049d60 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.592337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ff40-0037-4795-aef1-7329a2049d60 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.592986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 499/984] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:43.593749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 486/985] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:43.599094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.599386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.599668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.618084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:43.619336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:43.619670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ff40-0037-4795-aef1-7329a2049d60 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.646143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:43.700892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 500/986] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:43.722900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.750337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ff40-0037-4795-aef1-7329a2049d60 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:43.759205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:43.783136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:43.783934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.784220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.784492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.785136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:43.786538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:43.799746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:43.801078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:43.801919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 487/987] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:43.806713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.833271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a55d2e-01bf-43f2-b325-5b915d21edb1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:43.869401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:43.870533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.871014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.872114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] PATH_INFO: `/v3/services/image` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.873072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:43.874495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:43.888486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:43.889605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:43.890591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 501/988] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:43.895762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:43.913626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a69775-680c-4295-b277-8e95e2aeddd3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:43.917603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:43.950787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:43.951512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:43.951756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:43.952030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:43.952739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:43.953026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:43.954402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:43.959594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 488/989] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:43.965472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:44.001552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416143-5a6e-4c7b-8c4d-66ce3c7e8c2c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.003081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:44.036657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.037329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.037664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.037890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.038631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:44.039503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.040306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.050329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 502/990] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:44.051515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:44.074972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8b3c-1dc5-4c7d-ad06-df948ac930b2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.113008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.113820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.114136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.114418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.115119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:44.115424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.116772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.121678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 489/991] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:44.127376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:44.146570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:44.163511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469dd32-2d7a-4dd1-bae8-ff58941b321f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.201285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.202086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.202424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.202697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.203401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] RBAC: Adding query filter params (type=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:44.203824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.205193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.213430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 503/992] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:44.235355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:44.238628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b0a2f-2d26-48a3-a18a-1a30d0c259e3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.275948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.276534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.276815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.277086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.278475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] RBAC: Adding query filter params (interface=admin, service_id=3f547e48c9d34461a2e2eb3c2e7e7868, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:44.278793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.280197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f6dac-ff4e-4af7-9119-7f9496cd0018 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.290506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 490/993] 10.222.0.16 () {56 vars in 1249 bytes} [Mon May 23 20:05:44 2022] GET /identity/v3/endpoints?service_id=3f547e48c9d34461a2e2eb3c2e7e7868&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:44.292135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:44.397510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ababbe-5127-4bfc-8d73-0c0e6c4c5192 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.434381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.435071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.435352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.435622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.436319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.437747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333d5a5-dd69-49c0-823b-e0abe1a00fc1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.445157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 504/994] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:44 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:05:44.627684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f879d-2c84-4dc0-9c7d-6305dfec4896 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.627981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f879d-2c84-4dc0-9c7d-6305dfec4896 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.628265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f879d-2c84-4dc0-9c7d-6305dfec4896 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.629784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 491/995] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:44.634794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.635089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.635370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.652232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:44.653452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:44.661054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:44.720507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 505/996] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:44.743013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f879d-2c84-4dc0-9c7d-6305dfec4896 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:44.843038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f879d-2c84-4dc0-9c7d-6305dfec4896 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:44.876904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:44.877783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:44.878024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:44.878306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:44.878956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:44.880368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:44.892232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:44.893295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:44.894627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:44.898098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 492/997] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:44.899527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:45.000776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484be222-fb73-484b-a3cb-53fba3a137f3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:45.034347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:45.035005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:45.035280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:45.035555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:45.036255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:45.036556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:45.037925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:45.042830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 506/998] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:45.048692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:45.077248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:45.147529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e76b-f5f5-4ab3-9d23-e2d338d7984c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:45.181524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:45.182212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:45.182520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:45.182762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:45.183461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:45.183791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:45.185133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:45.189983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 493/999] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:45.195502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:45.204214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:45.305338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264631b-c730-4147-aca6-7e53dbed7496 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:45.340842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:45.341498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:45.341777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:45.342070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:45.342855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] RBAC: Adding query filter params (interface=public, service_id=a5aab37baeb140d699527b2f73cbf92f, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:45.343163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:45.344529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da867509-a757-49ba-b1bd-0492b98cbae5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:45.349675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 507/1000] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:45 2022] GET /identity/v3/endpoints?service_id=a5aab37baeb140d699527b2f73cbf92f&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:45.355136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:45.447728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:45.457214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84622bb8-86d8-4dfd-84a4-aceff0b93dea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:45.491503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:45.492169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:45.492451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:45.492725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:45.493403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:45.494785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13906c3-8d02-41e0-92bd-21dd0ae5fbf3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:45.501148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 494/1001] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:45 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:05:46.410995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042d9f5-b6c6-42f0-b08d-d402f4454b19 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.411526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042d9f5-b6c6-42f0-b08d-d402f4454b19 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.411526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042d9f5-b6c6-42f0-b08d-d402f4454b19 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.413041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 508/1002] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:46.417945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f3fd-cf4d-4036-b334-0ca1ea03b485 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.418357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f3fd-cf4d-4036-b334-0ca1ea03b485 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.418514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4f3fd-cf4d-4036-b334-0ca1ea03b485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.435151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-55e4f3fd-cf4d-4036-b334-0ca1ea03b485 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:46.436311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-55e4f3fd-cf4d-4036-b334-0ca1ea03b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:46.507474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 495/1003] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:46.529947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042d9f5-b6c6-42f0-b08d-d402f4454b19 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:46.599743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:46.633060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042d9f5-b6c6-42f0-b08d-d402f4454b19 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:46.667020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:46.667682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.668527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.668806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.669534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:46.671020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:46.672009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6b56c-f2f9-45b8-96b8-0ae246d43ca2 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.672300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6b56c-f2f9-45b8-96b8-0ae246d43ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.672583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6b56c-f2f9-45b8-96b8-0ae246d43ca2 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.674250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 496/1004] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:46.679666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.683611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.683611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.696980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:46.698243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:46.764934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 497/1005] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:46.773224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 509/1006] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:46 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:46.788538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:46.807872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:46.898088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e98fa-5f13-409f-835c-86492eabff8f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:46.941713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:46.942909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:46.943390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:46.943837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:46.944670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:46.946400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:46.959956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:46.960977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:46.961977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 498/1007] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 20:05:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:46.968218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:47.070050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc0752-51d6-4980-a465-cb913657ec98 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:47.099165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:47.105750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:47.106545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.112725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.112725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.112725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:47.112725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:47.112725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:47.115345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 510/1008] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:47.121695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:47.203881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:47.233865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdef7a-a7a5-4eef-8ff9-09da8a1cc848 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:47.270294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:47.271027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.271310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.271633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.272351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:47.272658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:47.274049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f3b43-31be-4b25-8181-666ada858868 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:47.283760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 499/1009] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 20:05:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:47.285063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:47.387811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0d80d-b64f-41ff-b5c5-c246c1273709 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:47.421629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:47.422318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.422587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.422863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.423618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:47.425000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d08aca-a146-44db-afe7-500d93226321 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:47.505604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 511/1010] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:47.706923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf160-a5f6-4ae5-902d-92214f1cc24a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.707416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf160-a5f6-4ae5-902d-92214f1cc24a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.707416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cf160-a5f6-4ae5-902d-92214f1cc24a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.708912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 500/1011] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:47.714076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.714363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.714810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.732534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:47.733728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:47.783750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:47.806964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 512/1012] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:47 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:47.829716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cf160-a5f6-4ae5-902d-92214f1cc24a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:47.932757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cf160-a5f6-4ae5-902d-92214f1cc24a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:47.967401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:47.968122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:47.968435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:47.968707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:47.969363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:47.970818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:47.983365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:47.984557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:47.985621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 501/1013] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:47.990478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:48.089447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924da10e-e313-4a3f-95fc-1d60f77e7939 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:48.123771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:48.124436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:48.124719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:48.125000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:48.125718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:48.126036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:48.127428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:48.132409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 513/1014] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:48.137987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:48.142751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:48.241442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e40e-83da-40c5-ba63-7ee2953327e8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:48.276129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:48.276804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:48.277087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:48.277911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:48.278647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:48.278959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:48.280314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3565bf4-b6c2-4191-beaf-5243bd66a704 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:48.289708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 502/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 20:05:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:48.291023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:48.339076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:48.393075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1c63-0682-4625-9a89-71db77bf53c5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:48.427209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:48.427927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:48.428146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:48.428417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:48.429150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:48.430552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1951897-055c-4959-b33f-381f49fbd44a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:48.541073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 514/1016] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:49.023061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5347-ac6d-4051-95b9-ab132c6973ab None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.023615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5347-ac6d-4051-95b9-ab132c6973ab None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.023615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5347-ac6d-4051-95b9-ab132c6973ab None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.025116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 503/1017] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:49.030301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.030587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.030857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.047799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:49.048957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:49.123364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 515/1018] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:49.145644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da5347-ac6d-4051-95b9-ab132c6973ab None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.217043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:49.254313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da5347-ac6d-4051-95b9-ab132c6973ab None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:49.287046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:49.287763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.288039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.288309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] PATH_INFO: `/v3/services/image` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.288962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:49.290342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:49.302369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:49.303381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:49.304594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:49.306093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 504/1019] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:49.310443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.336607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:49.409659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea659a5-1731-4892-a866-cbd6d41ec93b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:49.444349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:49.445008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.445281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.445548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.446406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:49.446712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:49.448083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:49.457680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 516/1020] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:49.458894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.564085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d75250-e02a-46a6-9aa4-d9478e5f4c54 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:49.593714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:49.599250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:49.600010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.600289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.600592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.604340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] RBAC: Adding query filter params (type=image) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:49.604340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:49.604340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885da7-347d-4398-8aa8-dd1969c03faa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:49.613646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 505/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:49.615200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.705785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:49.715969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1d433-ac0e-4ff9-9009-aed940c9f064 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:49.734400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21ec7-d4c4-4ff4-9ab7-d9c4400057ca None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.734730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21ec7-d4c4-4ff4-9ab7-d9c4400057ca None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.734958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca21ec7-d4c4-4ff4-9ab7-d9c4400057ca None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.736678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 506/1022] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:49.742480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.742809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.743030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.753350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:49.754009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.754306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.755026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.755824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] RBAC: Adding query filter params (interface=public, service_id=0a797c2c066e404cb2fbd5565a7c0196, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:49.756136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:49.757504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:49.759879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:49.761106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:49.763828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 517/1023] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/endpoints?service_id=0a797c2c066e404cb2fbd5565a7c0196&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:49.768903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.846419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 507/1024] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:49.869579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:49.872662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ac4d-811d-48f1-8b72-e08510322676 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:49.908644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:49.909298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:49.909580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:49.909855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:49.910569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:49.911956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:49.919474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 518/1025] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:05:49.979690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860e651-3923-4b20-9db2-7826ce8d8b14 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:50.008857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:50.018587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:50.019408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:50.019695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:50.019970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:50.021262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:50.022754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6304d-dc5c-4543-8da1-c8d4a20d8ec1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:50.024794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 508/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:50.030920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:50.052495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:50.140535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58efaa-7b41-40cd-bb5b-11eb033bc4a8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:50.175636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:50.176288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:50.176563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:50.176890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:50.177625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:50.179160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18e817-bf39-4543-888e-8f6f9f1fa07d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:50.259937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 519/1027] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:05:50 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:50.847325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc5b3-c002-479c-8701-c27947a223e9 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:50.847702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc5b3-c002-479c-8701-c27947a223e9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:50.847902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcc5b3-c002-479c-8701-c27947a223e9 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:50.849402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 509/1028] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:50.854774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:50.855203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:50.855447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:50.872803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:50.873980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:50.936602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:50.941666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 520/1029] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:50.965576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcc5b3-c002-479c-8701-c27947a223e9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.071072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcc5b3-c002-479c-8701-c27947a223e9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.104963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.105733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.106008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.106310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.116412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:51.125238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:51.126408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:51.132247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 510/1030] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:51.134480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.240976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15475a-5182-4292-bf7f-6e3fe48fcf68 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.276059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.276723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.277005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.277309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.278077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:51.278384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:51.279755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:51.292402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 521/1031] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:51.294277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.350046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:51.394455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f311f-c703-45b8-967c-d56df1851e55 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.431219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.432019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.432298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.432567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.443618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:51.445044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:51.461770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:51.462749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:51.468463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:51.468463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:51.468463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:51.468463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:51.468463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 511/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:51.470176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.531016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:51.570468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4a563-13b6-405f-a9d4-bd989c2ad924 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.603491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.604152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.604430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.604703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.606055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:51.606366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:51.607723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:51.623844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 522/1033] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:51.625674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.723783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febfb92-0538-499d-b0d0-1434135a77b1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.731413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:51.757457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.758285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.758567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.758836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.766036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:51.767734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:51.782555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:51.783521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:51.789247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 512/1034] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:51.791352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:51.889972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb25-b509-49f4-9f1f-28df8363cb9a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:51.912892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:51.926318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:51.927022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:51.927300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:51.927575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:51.928433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:51.928740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:51.930255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b65775-a571-41a5-aea7-d0a6b259473c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:51.954082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 523/1035] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:51.956751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.055247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98819-b41d-4ca2-93ca-dbdece24d2a8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.090284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.090940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.091202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.091472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.117447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49bd4b-f3f2-4f79-bd65-ef96d1259682 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.117747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49bd4b-f3f2-4f79-bd65-ef96d1259682 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.117999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49bd4b-f3f2-4f79-bd65-ef96d1259682 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.119531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 524/1036] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:52.124383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.124676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.124952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.125056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.142416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:52.143602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:52.161541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.163021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.219106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 525/1037] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:52.238175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.284582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 513/1038] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 20:05:52.285978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.345108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772da93a-2b75-47ba-9d6f-3dab590d0487 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.357435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.358073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.381543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.382447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.382819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.383094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.383744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.385117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.393027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13552-c422-42ab-9fee-ae39d1189446 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:52.397325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:52.398615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 20:05:52.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:52.407445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 526/1039] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:52.408308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.427171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.427963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.428244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.428517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.440205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.441636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:52.459084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:52.459998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:52.460878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:52.460878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:52.460878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:52.460878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:52.460878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 514/1040] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:52.465957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.520008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ba1ee-9de4-438a-b065-18d8da7904d1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.551952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.552605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.552891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.553159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.553854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:52.554188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.555544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbc8a1-b408-428c-9aa1-8d287ee0e3ed None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.561001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 527/1041] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:52.562043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddec3a-9536-4f06-8451-c8c6f983d028 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.562329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddec3a-9536-4f06-8451-c8c6f983d028 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.562623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddec3a-9536-4f06-8451-c8c6f983d028 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.562769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041cc7-8bd2-4d25-bec8-5bbbcd7ef1d7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.566056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 528/1042] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:52.566264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddec3a-9536-4f06-8451-c8c6f983d028 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.582490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.605706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.605706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.605706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.605706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.606111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:52.606111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.606111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c7c1-6470-46fa-b03e-d9e2b1966c6c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.606302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.614283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 515/1043] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:52.615490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.615784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.616064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.633014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:52.634407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:52.666220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddec3a-9536-4f06-8451-c8c6f983d028 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.698806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 516/1044] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:52.699999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.700235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.700893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.701176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.701450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.702198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:52.702506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.703876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.713135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 529/1045] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:52.714137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.801952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78e22f-4e62-49ee-b732-a24dda7d42b8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.811435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.815390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63630c8-297f-40df-9688-9db1f53570e1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:52.816899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:52.834840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.835634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.835916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.836193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.842637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.844074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.847905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:52.848664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:52.849004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:52.849359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:52.850118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:52.851721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:52.899977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:52.901653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 517/1046] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:52.902845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.936423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 530/1047] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:05:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:05:52.937308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:52.974183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d2c7-6af1-4752-b70e-18eca73e601c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:53.021498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:53.022326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:53.022611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:53.022881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:53.024120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:53.025622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:53.027603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 518/1048] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:53.033438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:53.052161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4fd3d-baa7-4dae-9fb9-005ea958df7c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:53.067347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:53.092351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:53.093026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:53.093305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:53.093576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:53.094541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:53.094851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:53.096331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:53.115120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 531/1049] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:53.116791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:53.138584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:53.149553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d214-abcf-43ec-911e-ca03f81c4904 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:53.186530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:53.187138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:53.187422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:53.187701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:53.188412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:53.189824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40caa8c3-ca19-49aa-9288-35cd2d334533 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:53.227882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1ac3-2785-45ae-b362-6275be601f4b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:53.261371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:53.262202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:53.262728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:53.263006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:53.263835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] RBAC: Adding query filter params (role.id=fc9526fa70d14df89002a67b4f550d59, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:53.264151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:53.265693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a64dc-25e0-4858-8b6d-32d0e3393492 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:53.277868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 519/1050] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:05:53 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:05:53.278120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 532/1051] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:53 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=fc9526fa70d14df89002a67b4f550d59 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:55.155951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03777-e948-4399-9aa1-2a6aac1711c0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.156474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03777-e948-4399-9aa1-2a6aac1711c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.156474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03777-e948-4399-9aa1-2a6aac1711c0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.158037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 520/1052] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:55.163131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac472-153a-4ae2-b944-a569266d86b5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.163515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac472-153a-4ae2-b944-a569266d86b5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.163694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac472-153a-4ae2-b944-a569266d86b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.180301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5e5ac472-153a-4ae2-b944-a569266d86b5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:55.181478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5e5ac472-153a-4ae2-b944-a569266d86b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:55.253670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 533/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:55.276247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e03777-e948-4399-9aa1-2a6aac1711c0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.368708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.383550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e03777-e948-4399-9aa1-2a6aac1711c0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:55.422231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:55.422815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.423100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.423383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.440149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510533-776f-46d1-8709-313863a8cc46 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.440437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510533-776f-46d1-8709-313863a8cc46 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.440713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510533-776f-46d1-8709-313863a8cc46 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.445525 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 534/1054] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:55.447027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.447316 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.447593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.466306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:55.467504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:55.476101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.499914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:55.501359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b81a3-fc4c-4475-a1f2-f733ed56bb95 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:55.538222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 535/1055] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:55.560834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.593599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.606103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 521/1056] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:05:55.607474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a1abd-145d-49ac-9652-70d041fc5278 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.607812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a1abd-145d-49ac-9652-70d041fc5278 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.608225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a1abd-145d-49ac-9652-70d041fc5278 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.609505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 522/1057] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:55.614476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a1abd-145d-49ac-9652-70d041fc5278 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.660059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e7e47-9006-47bc-b152-041b30ae1505 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:55.693195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:55.693938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.694247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.694524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.704833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:55.712888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a1abd-145d-49ac-9652-70d041fc5278 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6bcb0628-be71-4dbb-bb5b-e0484e4287a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:55.713019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:55.713965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:55.715276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 536/1058] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:55.716140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.716140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.716140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.720527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.732899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:55.734144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:55.752830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:55.753630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.753912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.754228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.764038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:55.765470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:55.782512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:55.783453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:55.784375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:55.784375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:55.784375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:55.784375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:55.784375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 523/1059] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:55.784667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.802249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 537/1060] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:55.803070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.836208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.893329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82895c-eafc-458c-9b76-482484d273cc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:55.898784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7bcb98-b7db-41f5-9756-0f825b7bbd1d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:55.926971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:55.928409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.928409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.928409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.928849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:55.929047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:55.929356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:55.930742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:55.935002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:55.935687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:55.935975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:55.936667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:55.937424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:55.937734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:55.939134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:55.939256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 524/1061] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:55.940077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:55.962236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 538/1062] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:55.962236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.052467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067356-c662-4f42-a168-4b0c74ff179b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.054761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.060612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a9539-4faf-4904-a10d-04c37c72622e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.092501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.093316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.093635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.093864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.097131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.097936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.098241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.098510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.107508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:56.110924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.112487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:56.116013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:56.116950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:56.118053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 525/1063] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:56.123573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:56.133325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:56.134303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:56.139740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:56.139740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:56.139740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:56.139740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:56.139740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 539/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:56.141297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.194571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.248356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7a7c-e7ff-4e0b-ab33-74193ebc233c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.256168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630fb9a-b666-42f5-ab92-09606605f1bf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.289024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.289694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.289977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.290284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.291034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:56.291342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.293353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.301386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.301938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 526/1065] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:56.302143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.302428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.302749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.303429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:56.303729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.305088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.307324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.317037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 540/1066] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:56.322356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.332200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.405235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3ca4-9412-4095-b1ae-905844fb7129 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.419306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c249df-a4aa-420d-be8b-1f6d002b8c2c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.421537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.439100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.439977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.440266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.440537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] PATH_INFO: `/v3/users/705ab820e85f466bb62d032c9c3700bc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.452115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.452896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.453179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.453451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.453668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=705ab820e85f466bb62d032c9c3700bc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.455130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.456769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 527/1067] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/users/705ab820e85f466bb62d032c9c3700bc => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:56.460661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.462285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.462408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:05:56.477435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:56.478544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:56.479497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 541/1068] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:56.484608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.544548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.561351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140f9e1-2332-4ea1-b782-6793e156b3d9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.580803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641766a-c0ff-4c04-923f-b57fb7e18eb1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.596599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.597248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.597533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.597807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.617325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.617980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.618285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.618559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.619386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:56.619695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.621155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.639086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 542/1069] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:56.644536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.664492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.665923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.667836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.741322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfe8a2-ee97-4722-ae31-f65d5acd199e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.743385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.762080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 528/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 20:05:56.767595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.775433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.776215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.776496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.776766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.787266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.788709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:05:56.805694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:56.806769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:56.807664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:05:56.807664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:05:56.807664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:05:56.807664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:56.807664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 543/1071] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:05:56.812320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.865477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577499c3-0cbd-492b-94dd-d9805dda9818 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.884343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.899091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.899946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.900226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.900498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.901654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.903142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.904811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 529/1072] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:56.908829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89104710-874f-416f-84aa-e5fe39547bc3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:56.910850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:56.942755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:56.943407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:56.943684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:56.943957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:56.944668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:56.944973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:56.946353 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:56.957922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 544/1073] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:56.961855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:56.963491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:57.011935 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74b378-477c-4816-9457-c2d06177b843 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:57.043601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:57.044327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:57.044617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:57.044895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:57.045720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=fddb0bcb0375497480a3b51e0a443231, user.id=705ab820e85f466bb62d032c9c3700bc) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:57.046059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:57.047493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:57.052873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 530/1074] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/role_assignments?user.id=705ab820e85f466bb62d032c9c3700bc&scope.project.id=fddb0bcb0375497480a3b51e0a443231&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:57.059186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64403e27-db68-4f16-8ebb-54a5111dd392 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:57.095877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:57.096684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:57.096962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:57.097230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:57.099410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:57.105302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:57.106820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:05:57.122592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:57.123602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:57.124484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 545/1075] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:57.129819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:57.237042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6e2f3-ef1c-488e-acf3-0687a639f59c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:57.251339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:57.270227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:57.270836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:57.271117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:57.271390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:57.272173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:57.272483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:57.273894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4311c-c987-4ac5-ae26-094d6eccab4b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:57.285836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 531/1076] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:57.291711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:57.388899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a55c-2cb7-4c1b-8623-d385e667b1bd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:57.421645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:57.422572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:57.422852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:57.423124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/fc9526fa70d14df89002a67b4f550d59` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:57.425315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=4cf871584be84949a82a9da23fb16153, role_id=fc9526fa70d14df89002a67b4f550d59)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:57.426781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e44ef6-5e05-48f6-a6bc-899004fb9f3a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:57.498494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 546/1077] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:57 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/4cf871584be84949a82a9da23fb16153/roles/fc9526fa70d14df89002a67b4f550d59 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:05:59.206578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1bd6-5f87-4279-bd8c-df4ec3ebe28a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.207131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1bd6-5f87-4279-bd8c-df4ec3ebe28a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.207131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1bd6-5f87-4279-bd8c-df4ec3ebe28a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.208490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 532/1078] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:05:59.213819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.214833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.214918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.232407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:59.233596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:59.282031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:59.299053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 547/1079] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:05:59.321360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1bd6-5f87-4279-bd8c-df4ec3ebe28a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.419938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1bd6-5f87-4279-bd8c-df4ec3ebe28a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:59.451300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:59.452088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.452371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.452643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.463767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:59.472169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:59.473210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:05:59.474101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 533/1080] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:59.480635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.577810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb499f2-f000-4035-8532-d8e9697090fe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:59.610958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:59.611625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.611907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.612181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.612924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:05:59.613229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:59.614643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b9f34-1c15-4e9d-ac23-41e59acc77a8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:59.622630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 548/1081] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:59.628426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.655992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ccf2-4bd6-4ce4-ace4-89559b5c0f2a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.656277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ccf2-4bd6-4ce4-ace4-89559b5c0f2a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.656552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ccf2-4bd6-4ce4-ace4-89559b5c0f2a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.657914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 549/1082] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:05:59.662756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.663038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.663313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.663723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:59.679311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:05:59.680515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:05:59.722348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:59.729279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e198-5e35-4e93-94f0-6d94295a6801 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:59.746504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 550/1083] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:05:59.763152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:59.764025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.764309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.764583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] PATH_INFO: `/v3/users/705ab820e85f466bb62d032c9c3700bc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.765764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] RBAC: Authorizing `identity:get_user(user_id=705ab820e85f466bb62d032c9c3700bc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:59.767217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:59.768827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 534/1084] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/users/705ab820e85f466bb62d032c9c3700bc => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:05:59.770377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.774148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.868383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaf54b-a767-4476-a243-aba4fdea8338 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:59.870162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb92e6-f4b6-46a5-8b16-ca22c530b36c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:05:59.872197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:05:59.901885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:59.902719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.903001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.903274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.904005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:05:59.904852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:05:59.905135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:05:59.905407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:05:59.906620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:05:59.909065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:05:59.911086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 535/1085] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:05:59.913627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:05:59.916564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:05:59.921968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:05:59.922992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 20:05:59.923914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:05:59.926281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 551/1086] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:05:59.928891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:05:59.963815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:00.016604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1197859-5aab-4226-a429-48142cd27d6f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.029324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816a7fa-1b31-459b-9803-38b13ff11030 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.048681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.049560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.049838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.050140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231/users/705ab820e85f466bb62d032c9c3700bc/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.052067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] RBAC: Authorizing `identity:create_grant(project_id=fddb0bcb0375497480a3b51e0a443231, user_id=705ab820e85f466bb62d032c9c3700bc, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.053482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.062525 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.063182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.063461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.065780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.065780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:00.065780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.066172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.074382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 552/1087] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:05:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:06:00.079910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:00.146916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 536/1088] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:05:59 2022] PUT /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231/users/705ab820e85f466bb62d032c9c3700bc/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:06:00.172181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:00.175864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b8f0-7b84-42f7-999a-dddcf33a8142 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.215288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.216149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.216400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.216678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.228463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.230472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:06:00.249535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:00.250541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:00.251428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:06:00.251428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:06:00.251428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:06:00.251428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:00.251428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 553/1089] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:06:00.256713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:00.289040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:00.359391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fdf7f-6ef4-477f-b6ac-d055b8e2195a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.393479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.394292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.394580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.394860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.395585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:00.395896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.397283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.408522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 537/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:00.413820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:00.512353 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa4630-6d2f-462a-909c-fb4e6cf28360 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.545411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.546160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.546439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.546707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.553463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.554931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:06:00.569387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:00.570537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:00.571643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 554/1091] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 20:06:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:00.576276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:00.664475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:00.674247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dafc9-712c-485d-9e5e-6018e381aebd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.706780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.707448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.707730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.708005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.708843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:00.709153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.710614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.727981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 538/1092] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 20:06:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:00.734215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:00.746175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:00.829119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367b15-54cd-4a7a-a6d0-9f91d46159d3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:00.862368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:00.863016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:00.863299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:00.863570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:00.928408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:00.929858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:00.940793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:01.025450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 555/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:01.031863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:01.125502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b77d-8db9-4cfe-8d35-57ea91b7085c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:01.157903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:01.158712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:01.158992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:01.159264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:01.168763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:01.170217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:01.178372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:06:01.187857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:01.193851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:01.194782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:06:01.194782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:06:01.194782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:06:01.194782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:06:01.194782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 539/1094] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:01.195055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:01.294236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec41417-2c08-46c2-a6ea-72e888b5a7fd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:01.304716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:01.327809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:01.328470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:01.328752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:01.329026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:01.329745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:01.330079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:01.331476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:01.343775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 556/1095] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:01.349205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:01.444022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c1ac-bca9-487e-8e34-a68bdc69c78f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:01.476313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:01.477094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:01.477374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:01.477646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:01.484140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:01.485577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:06:01.499454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:01.500318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:06:01.501233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 540/1096] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:01.506600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:01.604162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765de8c2-f96e-494a-9c2d-4d0e312a7c9e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:01.636171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:01.637313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:01.637593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:01.637867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:01.638687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:01.639011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:01.640420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4080-4b3c-47fb-aa11-5e9a731e7e47 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:01.652105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 557/1097] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:06:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:01.658439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:01.716279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:01.755772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8893e-2269-4146-8d2c-550786b5d2bf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:01.788612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:01.789264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:01.789544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:01.789818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:01.790678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] RBAC: Adding query filter params (role.id=fc9526fa70d14df89002a67b4f550d59, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=4cf871584be84949a82a9da23fb16153) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:01.790991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:01.792892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b71f07-4406-461b-b62e-5fb33615cebf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:01.798624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 541/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:01 2022] GET /identity/v3/role_assignments?user.id=4cf871584be84949a82a9da23fb16153&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=fc9526fa70d14df89002a67b4f550d59 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:02.355993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a54bf-8e19-450b-9912-17f56ffebfce None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:02.356546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a54bf-8e19-450b-9912-17f56ffebfce None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:02.356546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a54bf-8e19-450b-9912-17f56ffebfce None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:02.357892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 558/1099] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:02.363217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:02.363666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:02.363766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:02.380388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:02.381553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:02.449245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 542/1100] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:02.471717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a54bf-8e19-450b-9912-17f56ffebfce None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:02.502862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:02.576594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a54bf-8e19-450b-9912-17f56ffebfce None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:02.610322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:02.611106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:02.611381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:02.611647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:02.621590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:02.629080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:02.630116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:02.631051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 559/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:02.636803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:02.690508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:02.734444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70bc6-5f31-4c1f-a99e-3382f9ad05aa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:02.766635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:02.767247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:02.767534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:02.767812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:02.768563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:02.768872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:02.770249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:02.779068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 543/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:02.784744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:02.872749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:02.882064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f69a17-8fc6-434a-851d-f9acbb540ee7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:02.915700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:02.916555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:02.916836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:02.917108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] PATH_INFO: `/v3/users/705ab820e85f466bb62d032c9c3700bc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:02.918348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] RBAC: Authorizing `identity:get_user(user_id=705ab820e85f466bb62d032c9c3700bc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:02.919786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:02.921387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 560/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:02 2022] GET /identity/v3/users/705ab820e85f466bb62d032c9c3700bc => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:02.926905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:03.022077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7eea-c386-48a7-a567-9930892fc39c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:03.037312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:03.053702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:03.054366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:03.054652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:03.054927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:03.119097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:03.121032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:03.214972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 544/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:03.220227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:03.315531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5929-a025-4deb-8bff-00a8a00a62b2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:03.347259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:03.348103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:03.348381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:03.348650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:03.349793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:03.351507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad115de7-6215-436e-a2a3-eed6d58a0414 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:03.353171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 561/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:03 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:03.359642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:03.376924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:03.454543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ac52f-5016-44ae-89b5-a71a9a8d641d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:03.487011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:03.487653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:03.487932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:03.488205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:03.489015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=fddb0bcb0375497480a3b51e0a443231, user.id=705ab820e85f466bb62d032c9c3700bc) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:03.489324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:03.490755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464b026-11d3-47d1-aa56-f20606127e55 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:03.496878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 545/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:03 2022] GET /identity/v3/role_assignments?user.id=705ab820e85f466bb62d032c9c3700bc&scope.project.id=fddb0bcb0375497480a3b51e0a443231&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:06:05.690483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a5ee-5e50-4644-b334-8540ede1b8fa None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:05.691026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a5ee-5e50-4644-b334-8540ede1b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:05.691026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a5ee-5e50-4644-b334-8540ede1b8fa None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:05.692387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 562/1107] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:05.697595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:05.698033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:05.698165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:05.715284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:05.716448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:05.778997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 546/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:06:05.803030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072a5ee-5e50-4644-b334-8540ede1b8fa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:05.829696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:05.901427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072a5ee-5e50-4644-b334-8540ede1b8fa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:05.933808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:05.934617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:05.934900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:05.935168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:05.936345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:05.937772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a57f8-e4f3-440a-a9e2-7bc0bf4546f5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:05.939455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 563/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:05.945370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:05.957114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:06.040128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087773-5e28-46ba-9923-89fb0ada219f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:06.071863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:06.072515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:06.072793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:06.073059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:06.074377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:06.075779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6cf55-1163-47fb-86fe-eb11d6612d85 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:06.157236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 547/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:06:05 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:08.364523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed653-a705-4c0a-ad9e-4517ce2fa68a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:08.365050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed653-a705-4c0a-ad9e-4517ce2fa68a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:08.365050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed653-a705-4c0a-ad9e-4517ce2fa68a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:08.366476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 564/1111] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:08.371705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:08.372129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:08.372233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:08.388400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:08.389556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:08.445315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:08.452278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 548/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:08.475192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ed653-a705-4c0a-ad9e-4517ce2fa68a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:08.565653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:08.572837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ed653-a705-4c0a-ad9e-4517ce2fa68a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:08.606092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:08.606937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:08.607214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:08.607482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] PATH_INFO: `/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:08.608916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] RBAC: Authorizing `identity:get_role(role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:08.610357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:08.611691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 565/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:08 2022] GET /identity/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:08.617323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:08.709999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2b041-855c-4231-a752-4a3b07195a3e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:08.742186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:08.742980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:08.743257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:08.743529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] PATH_INFO: `/v3/users/77e339ba5a5049e0b43fa57479ea2e48` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:08.755750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] RBAC: Authorizing `identity:get_user(user_id=77e339ba5a5049e0b43fa57479ea2e48)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:08.757176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:08.758792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 549/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:08 2022] GET /identity/v3/users/77e339ba5a5049e0b43fa57479ea2e48 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:08.764708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:08.859287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b11b1-d858-44d1-ae81-f918ea7e5751 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:08.892015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:08.892640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:08.893143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:08.893420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:08.922526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:08.962004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:08.964998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:09.058684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 566/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:09.064405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:09.137461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:09.179849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d437b-ab64-4634-88e7-437accb43c6c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:09.211741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:09.212599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:09.212878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:09.213156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:09.214383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:09.215802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7e150-2180-4e3d-a3b8-8645e396c356 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:09.217463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 550/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:09 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:09.223375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:09.281693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:09.319083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d0c24-a16a-4ac1-b3c6-4fd30ec4fe6d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:09.351248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:09.351888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:09.352167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:09.352440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:09.353258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=fddb0bcb0375497480a3b51e0a443231, user.id=77e339ba5a5049e0b43fa57479ea2e48) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:09.353567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:09.355528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e42b1-1942-45fd-a92d-5cacce07e55b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:09.360707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 567/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:09 2022] GET /identity/v3/role_assignments?user.id=77e339ba5a5049e0b43fa57479ea2e48&scope.project.id=fddb0bcb0375497480a3b51e0a443231&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:11.505189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890303b0-cdc0-4d8f-9398-a3b767e92782 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:11.505686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890303b0-cdc0-4d8f-9398-a3b767e92782 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:11.505686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890303b0-cdc0-4d8f-9398-a3b767e92782 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:11.507069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 551/1118] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:11.512808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:11.513184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:11.513363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:11.530334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:11.531520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:11.595773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 568/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:11.617888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890303b0-cdc0-4d8f-9398-a3b767e92782 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:11.713529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890303b0-cdc0-4d8f-9398-a3b767e92782 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:11.735492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:11.746194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:11.747034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:11.747372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:11.747594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] PATH_INFO: `/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:11.749039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] RBAC: Authorizing `identity:get_role(role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:11.750424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:11.756173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 552/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:11 2022] GET /identity/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:11.757430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:11.810983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:11.852876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4804a1-2d2f-4469-8ef3-77ecaed7eaac None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:11.886410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:11.887238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:11.887516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:11.887785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] PATH_INFO: `/v3/users/77e339ba5a5049e0b43fa57479ea2e48` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:11.888956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] RBAC: Authorizing `identity:get_user(user_id=77e339ba5a5049e0b43fa57479ea2e48)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:11.890418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:11.892131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 569/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:11 2022] GET /identity/v3/users/77e339ba5a5049e0b43fa57479ea2e48 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:11.897668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:11.992363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c26644-e9b0-4cd7-8695-dffe65bc28f1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:12.025102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:12.025899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:12.026212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:12.026487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:12.027647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:12.029054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b1a97-94fd-4e26-84fb-b1b158ac5c9e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:12.030742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 553/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:11 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:12.036627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:12.131096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aa6-32d5-4a74-86da-6a3b22756cbc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:12.153623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:12.163728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:12.166486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:12.166486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:12.166486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231/users/77e339ba5a5049e0b43fa57479ea2e48/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:12.167344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] RBAC: Authorizing `identity:create_grant(project_id=fddb0bcb0375497480a3b51e0a443231, user_id=77e339ba5a5049e0b43fa57479ea2e48, role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:12.168758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6243e-28ce-4753-99b5-67919eb4c7da None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:12.232530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 570/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:06:12 2022] PUT /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231/users/77e339ba5a5049e0b43fa57479ea2e48/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:06:14.377987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9b3a-d876-43ce-af8b-9acdb1c34df3 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:14.378520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9b3a-d876-43ce-af8b-9acdb1c34df3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:14.378615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9b3a-d876-43ce-af8b-9acdb1c34df3 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:14.379944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 554/1124] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:14.385356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:14.385767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:14.385909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:14.402730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:14.403924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:14.468056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 571/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:14.491998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b9b3a-d876-43ce-af8b-9acdb1c34df3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:14.548210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:14.588387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b9b3a-d876-43ce-af8b-9acdb1c34df3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:14.620548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:14.621394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:14.621675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:14.621948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] PATH_INFO: `/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:14.623421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] RBAC: Authorizing `identity:get_role(role_id=eb3cd48f1d6740b89619fb8b76c2896c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:14.624771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:14.626144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 555/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:14 2022] GET /identity/v3/roles/eb3cd48f1d6740b89619fb8b76c2896c => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:14.631654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:14.726358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3b0dd-cc32-46ff-a6b0-1dce3dbc9815 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:14.739520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:14.758879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:14.759688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:14.759969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:14.760242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] PATH_INFO: `/v3/users/77e339ba5a5049e0b43fa57479ea2e48` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:14.761913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] RBAC: Authorizing `identity:get_user(user_id=77e339ba5a5049e0b43fa57479ea2e48)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:14.763447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:14.765053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 572/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:14 2022] GET /identity/v3/users/77e339ba5a5049e0b43fa57479ea2e48 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:14.770603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:14.865203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63871-e258-4790-b1bc-3b0e9543b221 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:14.889500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:14.897548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:14.902620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:14.902620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:14.902620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:14.961619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:14.963078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:15.054940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 556/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:15.061139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:15.156395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971460-bb4a-48cd-b0ef-4995bd7d8727 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:15.188801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:15.189642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:15.189925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:15.190225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] PATH_INFO: `/v3/projects/fddb0bcb0375497480a3b51e0a443231` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:15.191391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] RBAC: Authorizing `identity:get_project(project_id=fddb0bcb0375497480a3b51e0a443231)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:15.192812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc3eab-4fc0-44da-a034-29c18a356432 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:15.194491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 573/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:15 2022] GET /identity/v3/projects/fddb0bcb0375497480a3b51e0a443231 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:15.200939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:15.226610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:15.297023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834021be-82b5-45a9-bdbc-a19b68d502cc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:15.329788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:15.330474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:15.330758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:15.331031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:15.331851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] RBAC: Adding query filter params (role.id=eb3cd48f1d6740b89619fb8b76c2896c, scope.project.id=fddb0bcb0375497480a3b51e0a443231, user.id=77e339ba5a5049e0b43fa57479ea2e48) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:15.332159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:15.333570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee95ab-bb49-4f3a-b05e-b74923e26374 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:15.339299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 557/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:15 2022] GET /identity/v3/role_assignments?user.id=77e339ba5a5049e0b43fa57479ea2e48&scope.project.id=fddb0bcb0375497480a3b51e0a443231&role.id=eb3cd48f1d6740b89619fb8b76c2896c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:17.504239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c3dc-9c02-49fa-ab59-b238fbdaa0f1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:17.504738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c3dc-9c02-49fa-ab59-b238fbdaa0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:17.504738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c3dc-9c02-49fa-ab59-b238fbdaa0f1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:17.506167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 574/1131] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:17.511308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:17.511722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:17.511869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:17.528046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:17.529213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:17.592282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 558/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:17.615970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089c3dc-9c02-49fa-ab59-b238fbdaa0f1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:17.664515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:17.712842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089c3dc-9c02-49fa-ab59-b238fbdaa0f1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:17.745920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:17.746710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:17.746991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:17.747259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:17.748411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:17.749828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c208cea-6277-49fd-8193-88d3e0a0904d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:17.751430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 575/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:06:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:17.757118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:17.776971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:17.850946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc3450-7cee-4ebb-b170-5a3dff01e559 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:17.883540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:17.884195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:17.884481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:17.884756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:17.885461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:17.886979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef96052-5ab5-48c0-b2ef-a18acb66a25f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:17.964516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 559/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:06:17 2022] POST /identity/v3/projects => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:20.130519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adda0-856b-4199-94df-2775964c2e3a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:20.131118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adda0-856b-4199-94df-2775964c2e3a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:20.131118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701adda0-856b-4199-94df-2775964c2e3a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:20.132458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 576/1135] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:20.137611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:20.137986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:20.138194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:20.154839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:20.155998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:20.218950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 560/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:20.243536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701adda0-856b-4199-94df-2775964c2e3a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:20.341929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701adda0-856b-4199-94df-2775964c2e3a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:20.359795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:20.375566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:20.376411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:20.382416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:20.382416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:20.382416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:20.382416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa15fa4-8d85-4ef0-9795-38b8f10bff9e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:20.382416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 577/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:06:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:20.386563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:20.388675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:20.481112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88e942-56f7-4f67-b5a3-246442780919 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:20.512762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:20.513416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:20.513694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:20.513962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:20.514684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:20.516084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17919683-7b9b-477e-ba3d-4750b978820f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:20.594983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 561/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:06:20 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:22.813851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93a72-1ad3-401e-ade9-eba0068d9705 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:22.814404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93a72-1ad3-401e-ade9-eba0068d9705 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:22.814404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab93a72-1ad3-401e-ade9-eba0068d9705 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:22.815781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 578/1139] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:22.820891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:22.821329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:22.821458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:22.838405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:22.839592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:22.890703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:22.903386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 562/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:22.927182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93a72-1ad3-401e-ade9-eba0068d9705 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.024984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab93a72-1ad3-401e-ade9-eba0068d9705 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.057038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.058372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.058651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.058919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.068583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:23.076097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:23.077175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:23.078240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 579/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:23.083022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.177655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2240c-c11a-469f-9c58-e2495511cfee None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.209149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.209801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.210130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.210410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.211155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:23.211466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:23.212842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:23.221484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 563/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:23.227482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.272887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:23.324699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224d8b-c872-45e8-86af-1889a590059a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.357608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.358480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.358762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.359036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] PATH_INFO: `/v3/users/75d2bb0058a0477eb91c141da529c331` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.372000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=75d2bb0058a0477eb91c141da529c331)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:23.373435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:23.375075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 580/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:23 2022] GET /identity/v3/users/75d2bb0058a0477eb91c141da529c331 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:23.380820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.429180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:23.477671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6990b72-69d4-4353-ba08-8aa2e8dfdc0a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.509911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.511110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.511393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.511671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.576298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:23.577672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:23.623388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:23.672218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 564/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:23.678091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.772698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9da1-cd11-4527-b204-b66dd7652cf5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.774971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:23.804852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.805812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.806114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.806383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] PATH_INFO: `/v3/projects/2ac98553c1224627a03848b92a18eecd` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.807571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=2ac98553c1224627a03848b92a18eecd)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:23.808981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd7658-d51b-4e5b-b069-dad686294cd1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:23.810924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 581/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:23 2022] GET /identity/v3/projects/2ac98553c1224627a03848b92a18eecd => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:23.816578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:23.915682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96183474-0753-488d-8e38-edfc3647aa05 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:23.948617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:23.949265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:23.949549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:23.949827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:23.950682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=2ac98553c1224627a03848b92a18eecd, user.id=75d2bb0058a0477eb91c141da529c331) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:23.950996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:23.952413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c459253-0286-493c-a19c-c24e5b0e0168 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:23.957660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 565/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:23 2022] GET /identity/v3/role_assignments?user.id=75d2bb0058a0477eb91c141da529c331&scope.project.id=2ac98553c1224627a03848b92a18eecd&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:26.072334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fefef-2778-4cfa-9127-290876e6246b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.072860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fefef-2778-4cfa-9127-290876e6246b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.072860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fefef-2778-4cfa-9127-290876e6246b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.074282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 582/1147] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:26.079558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.079947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.080232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.097037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:26.098251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:26.107979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:26.163142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 566/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:06:26.185548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fefef-2778-4cfa-9127-290876e6246b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:26.283356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fefef-2778-4cfa-9127-290876e6246b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:26.316255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:26.317032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.317314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.317590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.327468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:26.334931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:26.336012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:26.336909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 583/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:26.342549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:26.436519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f5a14-5202-4da9-815d-34ec8e378ae0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:26.468735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:26.469387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.469668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.469940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.470701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:26.471006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:26.472362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:26.480970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 567/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:26.486571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:26.516805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:26.581643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30ab9f-c0ec-49ca-947c-eea064d39a5b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:26.614863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:26.615723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.616003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.616277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] PATH_INFO: `/v3/users/75d2bb0058a0477eb91c141da529c331` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.617437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] RBAC: Authorizing `identity:get_user(user_id=75d2bb0058a0477eb91c141da529c331)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:26.618889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:26.620501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 584/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:26 2022] GET /identity/v3/users/75d2bb0058a0477eb91c141da529c331 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:26.626083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:26.634981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:26.722142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c593e0-d15f-4d54-939f-946395e5b53d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:26.753413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:26.754289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.754572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.754844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] PATH_INFO: `/v3/projects/2ac98553c1224627a03848b92a18eecd` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.756002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] RBAC: Authorizing `identity:get_project(project_id=2ac98553c1224627a03848b92a18eecd)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:26.757407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6539288-87b9-40be-a837-2a51343b5988 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:26.759095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 568/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:26 2022] GET /identity/v3/projects/2ac98553c1224627a03848b92a18eecd => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:26.765223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:26.857540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:26.860294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba041-77fc-4c3c-bc67-704b1930176d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:26.893392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:26.894295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:26.894568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:26.894841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] PATH_INFO: `/v3/projects/2ac98553c1224627a03848b92a18eecd/users/75d2bb0058a0477eb91c141da529c331/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:26.896748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ac98553c1224627a03848b92a18eecd, user_id=75d2bb0058a0477eb91c141da529c331, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:26.898194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ffb5f-54bc-4af6-af9a-322623468db0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:26.997661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 585/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:06:26 2022] PUT /identity/v3/projects/2ac98553c1224627a03848b92a18eecd/users/75d2bb0058a0477eb91c141da529c331/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:06:29.128870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beff1a-6ba5-40f1-93db-df191efbfcf5 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.129424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beff1a-6ba5-40f1-93db-df191efbfcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.129424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beff1a-6ba5-40f1-93db-df191efbfcf5 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.130924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 569/1154] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:29.136100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.136488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.136650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.153331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:29.154550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:29.218809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 586/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:29.242522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beff1a-6ba5-40f1-93db-df191efbfcf5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:29.315098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:29.338994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02beff1a-6ba5-40f1-93db-df191efbfcf5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:29.371559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:29.372335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.372614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.372884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.382721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:29.390350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:29.391472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:06:29.392523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 570/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:29.397408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:29.483249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:29.492253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559440d9-e5e9-406f-b790-b3142ef52b91 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:29.524489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:29.525140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.525422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.526287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.527037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:29.527347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:29.528711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:29.536370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 587/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:29.541897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:29.636978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9129d-1ff8-494f-8bdf-d50531188051 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:29.668563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:29.669413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.669694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.669967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] PATH_INFO: `/v3/users/75d2bb0058a0477eb91c141da529c331` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.671156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] RBAC: Authorizing `identity:get_user(user_id=75d2bb0058a0477eb91c141da529c331)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:29.672572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:29.674189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 571/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:29 2022] GET /identity/v3/users/75d2bb0058a0477eb91c141da529c331 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:29.679583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:29.774805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16482335-6eeb-4c4d-aec3-8e556996909b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:29.806673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:29.807335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:29.807615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:29.807886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:29.828188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:29.872909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:29.874380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:29.967906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 588/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:29.973555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:29.976621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:30.070139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e081a60-95f0-4bd9-a536-d5817ed762b9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:30.102413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:30.103264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:30.103550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:30.103853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] PATH_INFO: `/v3/projects/2ac98553c1224627a03848b92a18eecd` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:30.105020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] RBAC: Authorizing `identity:get_project(project_id=2ac98553c1224627a03848b92a18eecd)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:30.106487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed17fd-bf81-4c4b-9996-752aea1e62a2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:30.108153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 572/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:29 2022] GET /identity/v3/projects/2ac98553c1224627a03848b92a18eecd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:30.113947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:30.165381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:30.214045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c6cb4-88ec-4217-ae5b-64854c00283c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:30.244224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:30.245473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:30.245753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:30.246050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:30.246874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=2ac98553c1224627a03848b92a18eecd, user.id=75d2bb0058a0477eb91c141da529c331) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:30.247185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:30.248621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a170f1-4d54-4728-8663-6a267636126f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:30.254259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 589/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:30 2022] GET /identity/v3/role_assignments?user.id=75d2bb0058a0477eb91c141da529c331&scope.project.id=2ac98553c1224627a03848b92a18eecd&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:32.427650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2617db-9a66-4dcd-9ce6-a10a5a2b5156 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:32.428167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2617db-9a66-4dcd-9ce6-a10a5a2b5156 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:32.428289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2617db-9a66-4dcd-9ce6-a10a5a2b5156 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:32.429572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 573/1162] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:32.435031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:32.435397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:32.435587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:32.451766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:32.452946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:32.517530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 590/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:32.539733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2617db-9a66-4dcd-9ce6-a10a5a2b5156 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:32.610542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:32.641635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2617db-9a66-4dcd-9ce6-a10a5a2b5156 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:32.674074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:32.674708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:32.674986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:32.675257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:32.738641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:32.740075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:32.803431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:32.833128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 574/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:32.839584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:32.883908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:32.934831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935f9c5-0c2c-44f0-bf05-bab762e65b4f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:32.967352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:32.968130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:32.968408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:32.968680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:32.975662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:32.977135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-30a88d87-cb74-4c34-a0b7-a91aa7b7cd0e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:32.991113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:33.000369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:33.001345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 591/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:06:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:33.002252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:33.094278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f62b2-8b8f-4025-9d3e-a0c24477591d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:33.127403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:33.128066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:33.128347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:33.128620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:33.129346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:33.129652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:33.131037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ca4d0-cd9c-40a5-b35b-1fee92436382 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:33.138739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 575/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 20:06:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:35.286249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da18b3-8b91-4ed6-82f9-30c2764bc40b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:35.286747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da18b3-8b91-4ed6-82f9-30c2764bc40b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:35.286848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da18b3-8b91-4ed6-82f9-30c2764bc40b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:35.288149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 592/1167] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:35.294006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1bc-2c81-45f1-a011-b005b617e731 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:35.294410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1bc-2c81-45f1-a011-b005b617e731 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:35.294598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1bc-2c81-45f1-a011-b005b617e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:35.310742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2736d1bc-2c81-45f1-a011-b005b617e731 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:35.311924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2736d1bc-2c81-45f1-a011-b005b617e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:35.336837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:35.375033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 576/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:35.397241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da18b3-8b91-4ed6-82f9-30c2764bc40b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:35.495507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da18b3-8b91-4ed6-82f9-30c2764bc40b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:35.501908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:35.528286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:35.528924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:35.529208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:35.529484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:35.530273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:35.532180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbb957-94c6-48c6-9b12-57dd8af222a3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:35.597661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 593/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Mon May 23 20:06:35 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:37.788247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712e520-f233-4a38-b1c8-06257323c248 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:37.788793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712e520-f233-4a38-b1c8-06257323c248 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:37.788793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712e520-f233-4a38-b1c8-06257323c248 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:37.790181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 577/1170] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:37.796105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:37.796503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:37.796654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:37.813356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:37.814563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:37.877554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 594/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:06:37.900308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712e520-f233-4a38-b1c8-06257323c248 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:37.996999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712e520-f233-4a38-b1c8-06257323c248 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:38.028322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:38.029157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:38.029436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:38.029704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] PATH_INFO: `/v3/domains/69f6b67786d44c12a7245ceebd6ccb8b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:38.039613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=69f6b67786d44c12a7245ceebd6ccb8b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:38.041505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92821daa-5468-4325-a2a6-76faaabeff58 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:38.043113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 578/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Mon May 23 20:06:37 2022] GET /identity/v3/domains/69f6b67786d44c12a7245ceebd6ccb8b => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:38.048682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:38.137247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:38.142741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ed04-91d2-45ef-9d35-a8b1037108e0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:38.175738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:38.176378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:38.176657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:38.176928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:38.177626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:38.179078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911c610-eb2c-4f5d-a104-29a6fc9e6865 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:38.272715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 595/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:06:38 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:40.450985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194476c-7dd4-4b57-838f-f9fcf622d175 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:40.451521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194476c-7dd4-4b57-838f-f9fcf622d175 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:40.451521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194476c-7dd4-4b57-838f-f9fcf622d175 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:40.452885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 579/1174] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:40.458196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:40.458568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:40.458753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:40.475694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:40.476810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:40.539881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 596/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:40.564064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194476c-7dd4-4b57-838f-f9fcf622d175 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:40.589829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:40.662930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194476c-7dd4-4b57-838f-f9fcf622d175 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:40.695128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:40.695978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:40.696258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:40.696535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] PATH_INFO: `/v3/domains/69f6b67786d44c12a7245ceebd6ccb8b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:40.697728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=69f6b67786d44c12a7245ceebd6ccb8b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:40.699184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1543d-837e-4af7-aae4-b0c56f881c54 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:40.700771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 580/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Mon May 23 20:06:40 2022] GET /identity/v3/domains/69f6b67786d44c12a7245ceebd6ccb8b => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:40.706402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:40.782570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:40.802604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44697db9-ee76-4d96-a05c-5e356c3dfe17 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:40.835553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:40.836210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:40.836493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:40.836770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:40.837481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:40.839538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83f22a-137b-44be-8a9f-11017e242cbd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:40.902680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 597/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:06:40 2022] POST /identity/v3/users => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:06:43.124717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591903a-623c-45fd-820c-1dea926ed62b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.125270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591903a-623c-45fd-820c-1dea926ed62b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.125270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591903a-623c-45fd-820c-1dea926ed62b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.126664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 581/1178] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:43.131792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.132234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.132384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.148576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:43.149750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:43.213084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 598/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:43.236574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591903a-623c-45fd-820c-1dea926ed62b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:43.323683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:43.332655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591903a-623c-45fd-820c-1dea926ed62b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:43.365619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:43.366424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.366705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.366975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.376575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:43.384049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:43.385141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:06:43.386233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 582/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:43.391125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:43.451913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:43.488632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453709e4-38cf-4a75-9114-9a2dfc27bacf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:43.520637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:43.521294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.521577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.521849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.522615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:43.522921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:43.524284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:43.532475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 599/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:43.538211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:43.633296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb60f-ba7d-4de1-9e39-d001949894a6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:43.664743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:43.665595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.666495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.666778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] PATH_INFO: `/v3/users/0820dd22370b4363917aa14385b87ed4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.679097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] RBAC: Authorizing `identity:get_user(user_id=0820dd22370b4363917aa14385b87ed4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:43.680524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:43.682174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 583/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:43 2022] GET /identity/v3/users/0820dd22370b4363917aa14385b87ed4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:43.687521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:43.783523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705cf1d-ad32-412e-98bc-f9091d665f50 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:43.807070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:43.815167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:43.817217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:43.817217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:43.817217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:43.881755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:43.883255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:43.978769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 600/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:43.984463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:43.996202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:44.080442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2edc8-e4d3-4c9e-a1d9-25c9afb59572 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:44.112345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:44.113199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:44.113475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:44.113743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] PATH_INFO: `/v3/projects/16788440fd5944318423432981a38cc7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:44.114931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] RBAC: Authorizing `identity:get_project(project_id=16788440fd5944318423432981a38cc7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:44.116370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3569beb-1595-4587-94d7-ffb3f4e2e2ce None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:44.118037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 584/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:43 2022] GET /identity/v3/projects/16788440fd5944318423432981a38cc7 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:44.124379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:44.150130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:44.218217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0f2c2-10a2-47a6-a576-fe61c685fa0f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:44.251146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:44.251734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:44.252016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:44.252293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:44.253114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=16788440fd5944318423432981a38cc7, user.id=0820dd22370b4363917aa14385b87ed4) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:44.253424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:44.254865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd919f-aa27-4a4f-987a-5f12f6b2fcfb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:44.260072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 601/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:44 2022] GET /identity/v3/role_assignments?user.id=0820dd22370b4363917aa14385b87ed4&scope.project.id=16788440fd5944318423432981a38cc7&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:46.376923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737350e-45aa-41e8-b53c-2f9addde1940 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:46.377426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737350e-45aa-41e8-b53c-2f9addde1940 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:46.377536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737350e-45aa-41e8-b53c-2f9addde1940 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:46.378889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 585/1186] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:46.384295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:46.384679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:46.384850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:46.401040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:46.402970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:46.466526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 602/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:46.489183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737350e-45aa-41e8-b53c-2f9addde1940 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:46.564291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:46.585082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737350e-45aa-41e8-b53c-2f9addde1940 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:46.618274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:46.619046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:46.619325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:46.619598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:46.629194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:46.636746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:46.637887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:06:46.638870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 586/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:46.643855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:46.664362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:46.742305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8808-52b6-4b28-82c6-b05a23f860c3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:46.774602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:46.775263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:46.775549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:46.775828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:46.777428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:46.777929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:46.779489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:46.787428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 603/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:46.793509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:46.889536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098c341-e76a-4f37-a340-7f0a718d0aae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:46.921429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:46.922316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:46.922601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:46.922875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] PATH_INFO: `/v3/users/0820dd22370b4363917aa14385b87ed4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:46.924056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] RBAC: Authorizing `identity:get_user(user_id=0820dd22370b4363917aa14385b87ed4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:46.925469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:46.927100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 587/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:46 2022] GET /identity/v3/users/0820dd22370b4363917aa14385b87ed4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:46.932742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:47.012328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:47.029542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d80b1-dfcf-498c-9259-243301719ceb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:47.062010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:47.062879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:47.063160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:47.063432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] PATH_INFO: `/v3/projects/16788440fd5944318423432981a38cc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:47.064591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] RBAC: Authorizing `identity:get_project(project_id=16788440fd5944318423432981a38cc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:47.066036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c6318-8fd4-4d9d-bd0a-31f9fe805013 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:47.067693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 604/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:46 2022] GET /identity/v3/projects/16788440fd5944318423432981a38cc7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:47.073958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:47.082949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:47.169564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4a09b-9e63-4313-b15c-1a155df07826 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:47.202208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:47.203084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:47.203363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:47.203637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] PATH_INFO: `/v3/projects/16788440fd5944318423432981a38cc7/users/0820dd22370b4363917aa14385b87ed4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:47.205557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] RBAC: Authorizing `identity:create_grant(project_id=16788440fd5944318423432981a38cc7, user_id=0820dd22370b4363917aa14385b87ed4, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:47.206976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25741-d090-4afe-a44c-a16f6acb2815 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:47.273668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 588/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:06:47 2022] PUT /identity/v3/projects/16788440fd5944318423432981a38cc7/users/0820dd22370b4363917aa14385b87ed4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:06:49.427403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20519-8fc8-4afa-9e77-90a64df4ca71 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:49.427940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20519-8fc8-4afa-9e77-90a64df4ca71 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:49.427940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20519-8fc8-4afa-9e77-90a64df4ca71 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:49.429324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 605/1193] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:06:49.435046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:49.435413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:49.435601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:49.452296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:06:49.453454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:06:49.509151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:49.516681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 589/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:06:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:06:49.539153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20519-8fc8-4afa-9e77-90a64df4ca71 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:49.637999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20519-8fc8-4afa-9e77-90a64df4ca71 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:49.671563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:49.672337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:49.672618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:49.672893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:49.683296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:06:49.690858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:06:49.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:06:49.693112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 606/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:06:49.697827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:49.792686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936cb0f-f3be-478c-8864-c9bd1e31c445 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:49.825016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:49.825669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:49.825954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:49.826258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:49.827001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:49.827309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:49.828657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:49.836400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 590/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:49.841851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:49.855087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:49.939932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900db2c-2bac-415d-92e3-dd1bbda7502e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:49.971459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:49.972313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:49.972591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:49.972859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] PATH_INFO: `/v3/users/0820dd22370b4363917aa14385b87ed4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:49.974057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=0820dd22370b4363917aa14385b87ed4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:49.975751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:49.977346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 607/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 20:06:49 2022] GET /identity/v3/users/0820dd22370b4363917aa14385b87ed4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:49.983397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:50.039035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:50.079781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c85ff-c8a5-4d02-ab50-30546c9882f3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:50.112530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:50.113167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:50.113453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:50.113729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:50.178494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:50.179921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:50.234343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:50.274448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 591/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:06:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:06:50.280676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:50.356576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:06:50.376649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa4d97-be99-430b-8baa-b6346637bbb5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:50.409197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:50.410061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:50.410345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:50.410618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] PATH_INFO: `/v3/projects/16788440fd5944318423432981a38cc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:50.411770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] RBAC: Authorizing `identity:get_project(project_id=16788440fd5944318423432981a38cc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:50.413203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78a61b-4c52-4363-8e7f-a1b93b29d004 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:50.414874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 608/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 20:06:50 2022] GET /identity/v3/projects/16788440fd5944318423432981a38cc7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:06:50.421317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:06:50.514904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c84cb-8f7c-48c4-9f19-6ec8252135bf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:06:50.546807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:06:50.547448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:06:50.547725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:06:50.547995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:06:50.548811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=16788440fd5944318423432981a38cc7, user.id=0820dd22370b4363917aa14385b87ed4) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:06:50.549117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:06:50.550555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f660-e621-49bb-9e95-81893c6da1e4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:06:50.556079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 592/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:06:50 2022] GET /identity/v3/role_assignments?user.id=0820dd22370b4363917aa14385b87ed4&scope.project.id=16788440fd5944318423432981a38cc7&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:07:00.405170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=27 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:07:00.553671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:07.063815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f76a-2108-409a-b8d6-862858a03e6a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:07.064421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f76a-2108-409a-b8d6-862858a03e6a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:07.064421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2f76a-2108-409a-b8d6-862858a03e6a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:07.065778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 609/1201] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:07.071047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:07.071449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:07.071604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:07.088428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:07.089597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:07.155181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 593/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:07.179645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2f76a-2108-409a-b8d6-862858a03e6a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:07.280870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2f76a-2108-409a-b8d6-862858a03e6a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:07.313550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:07.314410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:07.314640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:07.314914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:07.325081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:07.326563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:07.343367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:07.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:07.345860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:07.345860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:07.345860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:07.345860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:07.345860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 610/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:07.350826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:07.444451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae6752-fed4-4726-9d5e-478e72bf6905 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:07.471803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:07.477459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:07.478142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:07.478424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:07.478698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:07.479412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:07.479710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:07.481114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9819-72e9-4674-ab54-d00f1c99a0c3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:07.491929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 594/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:07.498133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:07.549060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:07.594221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85988-3e7c-4dd3-970c-1092d8d45bc7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:07.626818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:07.627471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:07.627772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:07.628031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:07.628734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:07.630181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773e201-1f88-4334-94fe-195264497d6f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:07.733267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 611/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 20:08:07 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:08:09.909483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629768f7-720e-4c1e-a667-278066ef2384 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:09.910079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629768f7-720e-4c1e-a667-278066ef2384 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:09.910079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629768f7-720e-4c1e-a667-278066ef2384 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:09.911425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 595/1206] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:09.916835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:09.917255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:09.917405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:09.935491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:09.936682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:10.003611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 612/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:10.027180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629768f7-720e-4c1e-a667-278066ef2384 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.126466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629768f7-720e-4c1e-a667-278066ef2384 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.157974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.166875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.166875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.166875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.169316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:10.176964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:10.178075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:10.179327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 596/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:10.184272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.223961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:10.283043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6cc0a-d1f0-4f1b-9144-8723885e5073 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.315324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:10.322212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:10.326479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 613/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:10.332084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.380740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:10.429009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660921aa-eabb-45f9-baed-2efa7bdaf1ea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.459894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.460699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.460978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.461250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.472386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:10.477050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:10.491293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:10.492364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:10.493280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:10.493280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:10.493280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:10.493280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:10.493280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 597/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:10.498367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.592451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:10.595308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576f30e-4725-466d-8839-9acc186e5bed None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.627779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.628427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.628705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.629014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.629723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:10.630041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:10.631403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:10.642997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 614/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:10.648633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.744675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607e3c-21f0-4f19-917e-f2b99ed2af84 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.746087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:10.777141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.778610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.778886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.779152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.786142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:10.788199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:10.802773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:10.803903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:10.804949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 598/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:10.809709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:10.905937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c5779-013d-4a74-a7a7-491beb48ce0d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:10.937510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:10.938199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:10.938478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:10.938751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:10.939576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:10.939878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:10.941297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:10.958003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 615/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:10.963786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.060512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32034b8d-036c-4ee1-88ec-57226940610f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:11.094060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:11.094705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:11.094990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:11.095270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:11.124234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:11.161486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:11.162957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:11.259913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 599/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:08:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:08:11.265938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.285191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:11.364393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139ed0-5aad-44de-bee0-b0775cbe6531 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:11.398063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:11.398853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:11.399133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:11.399404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:11.409684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:11.411171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:11.428233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:11.429265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:11.430401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:11.430401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:11.430401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:11.430401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:11.430401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 616/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:11.435110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.494608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:11.532201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba1d28-4398-4d0d-85c7-20eee2498f2c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:11.564925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:11.565581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:11.565864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:11.566165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:11.566883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:11.567191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:11.568582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:11.580148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 600/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:11.585774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.654649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:11.683630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e324d2d-cebc-4ff2-8ad5-13a3c607b231 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:11.716763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:11.717542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:11.717823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:11.718124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:11.724750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:11.726301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:11.739516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:11.740553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:11.741502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 617/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:11.747052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.845628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15694f-392d-4ed7-ae6d-b626e1bb88d7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:11.867436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:11.879182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:11.879840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:11.880122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:11.880395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:11.881323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:11.881636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:11.883110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63b198-a6c0-4ed5-96b7-8b9584e1809c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:11.895339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 601/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:11.901899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:11.999622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afe1c1-4e20-46f4-b1ca-fa49e0ec1caa None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:12.026893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:12.032397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:12.033040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:12.033318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:12.033588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:12.034441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=01927a09fd62479a9640dccdda24f762) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:12.034750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:12.036234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65ce2b-8208-470c-984d-5aac8f07ecc8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:12.041343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 618/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:08:11 2022] GET /identity/v3/role_assignments?user.id=01927a09fd62479a9640dccdda24f762&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:14.190758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73b77b-a055-4f75-a9ee-61f4fc381105 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.191335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73b77b-a055-4f75-a9ee-61f4fc381105 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.191335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73b77b-a055-4f75-a9ee-61f4fc381105 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.192672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 602/1220] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:08:14.197952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.198381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.198532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.216202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:14.217398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:14.285285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 619/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:14.308646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73b77b-a055-4f75-a9ee-61f4fc381105 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:14.406591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73b77b-a055-4f75-a9ee-61f4fc381105 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:14.439503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:14.440285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.440566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.440889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.451256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:14.459052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:14.460503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:14.461630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 603/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:08:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:14.467047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:14.562744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95af3d-08c4-44d2-857a-1429b0cdecbc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:14.586586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:14.597626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:14.598571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.598852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.599125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.599867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:14.600177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:14.601542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:14.609149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 620/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:08:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:14.615289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:14.657436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:14.716327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48282d9-518b-4fbc-8b29-93154c795409 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:14.749509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:14.750322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.750605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.750880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.761406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:14.763482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:14.780082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:14.781191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:14.782315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:14.782315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:14.782315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:14.782315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:14.782315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 604/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:14.786958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:14.884359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39e078-b080-42fe-9831-7546aa8d064d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:14.916851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:14.917500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:14.917779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:14.918102 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:14.918812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:14.919163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:14.920518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:14.931736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 621/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:14.937855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.029007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:15.035269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15082070-df96-4511-b875-29e058e9dcc0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.067423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.068217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.068500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.068775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.076092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.078194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:15.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:15.093659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:15.101701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 605/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:15.102449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.146332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:15.197541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff169a7-5303-4e3c-ad6f-902379641907 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.232734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.233409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.233688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.233958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.234825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:15.235137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.236587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.253997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 622/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:08:15.259723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.356330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef269b5-e798-477b-b0cb-4974f731afc7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.389058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.389839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.390147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.390420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.400076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.401494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:15.418876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:15.422345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:15.423456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:15.423456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:15.423456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:15.423456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:15.423456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 606/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:15.425660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.520488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e5819-9441-4655-aa46-e62d8373f3de None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.522760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:15.553165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.553825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.554201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.554476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.555185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:15.555485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.556875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.567606 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 623/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:15.573162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.582003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:15.669261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d73d-731f-45b0-935e-99d974ba048d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.700182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.700955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.701233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.701504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.707808 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.709213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:15.723818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:15.724872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:15.725990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 607/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:15.731440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.827557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d05-d575-4ce7-b567-0cb2f8a67897 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:15.860039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:15.860684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:15.860960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:15.861234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:15.861996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:15.862329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:15.863789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3574c-a824-4250-ba6c-4485122695a0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:15.879286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 624/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:15.885402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:15.949244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:15.982505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a83cd-df64-4fe4-a850-9900d6e05b3e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:16.018486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:16.019379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:16.019660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:16.019936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/01927a09fd62479a9640dccdda24f762/roles/9d842d034f934208b14a614c457bd4ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:16.034153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=01927a09fd62479a9640dccdda24f762, role_id=9d842d034f934208b14a614c457bd4ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:16.035581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde080ce-dfe4-4e2f-8254-42a52c20ee2d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:16.202004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 608/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:08:15 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/01927a09fd62479a9640dccdda24f762/roles/9d842d034f934208b14a614c457bd4ad => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:08:18.326348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c892-4fbc-4410-bfc4-1f40489e7186 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:18.326862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c892-4fbc-4410-bfc4-1f40489e7186 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:18.326977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c892-4fbc-4410-bfc4-1f40489e7186 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:18.328295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 625/1233] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:18.333583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:18.334036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:18.334171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:18.351305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:18.352496 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:18.417140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 609/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 20:08:18.441075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c892-4fbc-4410-bfc4-1f40489e7186 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:18.466050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:18.539008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c892-4fbc-4410-bfc4-1f40489e7186 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:18.572051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:18.572836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:18.573113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:18.573379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:18.583953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:18.591820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:18.597656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:18.598998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 626/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 20:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:18.600729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:18.645891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:18.695751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf679b-60a0-4d48-a645-71db94316139 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:18.728894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:18.729558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:18.729843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:18.730149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:18.730926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:18.731231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:18.732589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:18.740257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 610/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 20:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:08:18.746512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:18.828779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:18.843945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167b5d5-24bb-42ec-9f6a-1e7cb559c620 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:18.877310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:18.878133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:18.878654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:18.878991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:18.889267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:18.893970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:18.908438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:18.909462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:18.910784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:18.910784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:18.910784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:18.910784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:18.910784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 627/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:18.915265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.013713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95ca6-607e-4b63-9314-36b3759f526a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.022132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:19.045903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.046590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.046874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.047147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.047918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:19.048226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.049575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:19.061580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 611/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:19.067361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.162724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59bb1-bb37-4a94-bd83-385d4a07e1e9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.196234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.197021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.197301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.197573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.200738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:19.205041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.206561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:19.221362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:19.222520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:19.223838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 628/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:19.228701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.331089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484dbdb-c4d6-4a0a-8791-67efbc788a8a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.367759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.368438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.368699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.369024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.369857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:19.370202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.372154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:19.394063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 612/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:19.396489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.497822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af377a-637a-4c00-801e-21ff1d66c0b2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.532433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.533078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.533357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.533694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.592533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:19.598995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.600441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:19.694362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 629/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:08:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:08:19.700335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.727012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:19.796207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1b989-a766-4a15-9f55-0ce3fff61d1f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.828825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.837443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.837443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.837443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.839896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.841315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:19.859211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:19.860271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:19.861519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:19.861519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:19.861519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:19.861519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:19.861519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 613/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:19.866187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:19.956298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:19.961695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce6d4e-323a-4f35-8ded-6b1468e5d47f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:19.995185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:19.995841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:19.996216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:19.996488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:19.997452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:19.997918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:19.999610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:20.010658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 630/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:20.016264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:20.090908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:20.111658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c50bd4-c0f4-4c93-8af7-336e98420681 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:20.144143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:20.144973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:20.145247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:20.145545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:20.151928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:20.153860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:20.166989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:20.168672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:20.169779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 614/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:20.174677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:20.274670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3237528-7cf9-46ae-b77a-a512da13a13d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:20.307765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:20.308490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:20.308758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:20.309030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:20.309810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:20.310144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:20.311567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebe9a1-893e-486b-a18b-b2beaf745c75 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:20.323114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 631/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:20.329650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:20.424672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e95c3d-473e-4cde-9186-d71ad7e340a0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:20.456250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:20.456890 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:20.457167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:20.457438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:20.458276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] RBAC: Adding query filter params (role.id=9d842d034f934208b14a614c457bd4ad, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=01927a09fd62479a9640dccdda24f762) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:20.458583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:20.459977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40416d3-e002-4e81-9c0d-03a781adaf0d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:20.466182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 615/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:08:20 2022] GET /identity/v3/role_assignments?user.id=01927a09fd62479a9640dccdda24f762&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=9d842d034f934208b14a614c457bd4ad => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:22.678744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400941-877b-4418-b5d6-0da857ff0d0c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:22.679261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400941-877b-4418-b5d6-0da857ff0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:22.679354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400941-877b-4418-b5d6-0da857ff0d0c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:22.680684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 632/1247] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:22.686504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:22.686932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:22.687063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:22.692028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:22.703630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:22.704806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:22.768237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 616/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:22.790877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400941-877b-4418-b5d6-0da857ff0d0c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:22.809372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:22.890442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400941-877b-4418-b5d6-0da857ff0d0c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:22.923445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:22.924222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:22.924499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:22.924767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:22.935404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:22.943006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:22.944104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:22.945380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 633/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:22.955896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.046076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5df62-1a1b-4de8-ad6d-64b63ae250fb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.073840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:23.079352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.080030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.080338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.080614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.081365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:23.081672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.083069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:23.090929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 617/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:23.096663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.171848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:23.193905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3bc5c-87b4-4a5e-940a-6a030f5ced96 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.226439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.227229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.227508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.227778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.239242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.240685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:23.258324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:23.259423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:23.260488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:23.260488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:23.260488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:23.260488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:23.260488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 634/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:23.266062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.363190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fbe4d-121e-4a8f-9b1f-068727f07d77 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.395116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.395774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.396085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.396359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.397069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:23.397371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.398741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:23.409873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 618/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:23.415398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.512871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590d08-3551-408b-b8bc-60024dd1a47d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.537902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:23.545961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.546839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.547128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.547403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.554652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.556187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:23.570763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:23.571874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:23.573043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 635/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:23.577938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.632420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:23.674682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abdc9-15ba-4907-8c7e-d2856d9f466c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.707075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.707746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.708029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.708313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.709143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:23.709457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.710974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:23.731075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 619/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:23.736695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:23.834269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aa332-cd79-496a-b49d-3901dd31731e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:23.866604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:23.867239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:23.867518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:23.867791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:23.922582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:23.934996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:23.936507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.033508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 636/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:08:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:08:24.039239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:24.137146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135511-4ad7-4db2-b361-e8326eebfc53 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:24.149690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:24.170128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:24.170937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:24.171216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:24.171487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:24.181878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:24.183968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:24.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:24.202452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:24.203558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:24.203558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:24.203558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:24.203558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:24.203558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 620/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:24.208397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:24.293454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:24.305062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294c93d-3760-4df3-b5af-1617f99e710a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:24.337232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:24.337880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:24.338185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:24.338456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:24.339164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:24.339463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:24.340857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.351806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 637/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:24.357973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:24.457811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0932b-b895-4d7d-97ac-a16772ad64ef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:24.491674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:24.492463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:24.492745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:24.493018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:24.499527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:24.501009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:24.514505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:24.515583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:24.516503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 621/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:24.521423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:24.621993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fd3c8-3ccb-4d34-b094-de49d3c71087 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:24.654447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:24.655099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:24.655391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:24.655665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:24.656443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:24.656786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:24.658221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b4380-f5df-4ad1-b953-87267f72aff2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.670324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 638/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:24.676313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:24.701107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:24.772007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb7e75-c270-4623-95d6-8fbb6562a811 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:24.806486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:24.807149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:24.807452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:24.807711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:24.808538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=01927a09fd62479a9640dccdda24f762) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:24.808848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:24.810305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2f41-a864-40a8-99fd-904ed87bcfc8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:24.815686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 622/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:08:24 2022] GET /identity/v3/role_assignments?user.id=01927a09fd62479a9640dccdda24f762&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:26.963973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5a34-5a11-4d69-b0ba-674d471e3b5c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:26.964568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5a34-5a11-4d69-b0ba-674d471e3b5c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:26.964568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5a34-5a11-4d69-b0ba-674d471e3b5c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:26.965902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 639/1261] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:26.971202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:26.973908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:26.973908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:26.988633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:26.989804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:27.055198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 623/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:27.078248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5a34-5a11-4d69-b0ba-674d471e3b5c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.088977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:27.175907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5a34-5a11-4d69-b0ba-674d471e3b5c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:27.210836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:27.211646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:27.211925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:27.212201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:27.222451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:27.230900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:27.231919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:27.232771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 640/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 20:08:27.238349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.257489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:27.335001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b161f44-3c1f-4d74-886f-bd7b9002d61c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:27.368136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:27.368796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:27.369079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:27.369355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:27.370120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:27.370426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:27.372448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:27.380287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 624/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:27.386297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.457161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:27.489977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c360ab-4540-400b-b977-05e6dcee842c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:27.521300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:27.522116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:27.522399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:27.522676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:27.534571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:27.536022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:27.553256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:27.554395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:27.555403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:27.555403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:27.555403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:27.555403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:27.555403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 641/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:27.560389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.638442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:27.656969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22160dad-336e-47b0-abbf-3a85c58fb195 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:27.690537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:27.691206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:27.691493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:27.691769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:27.692491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:27.692801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:27.694200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:27.706041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 625/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:27.711703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.810450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c198f-db23-4aef-bc60-6b260ceea190 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:27.829734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:27.843973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:27.844774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:27.845055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:27.845324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:27.853265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:27.855096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:27.869789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:27.871133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:27.872093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 642/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:27.876945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:27.974758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ca2e-ebda-4623-8d89-371eabc5d8f4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.006992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.007707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.007990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.008266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.009094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:28.009411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.010888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.020311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:28.029660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 626/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:28.035587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:28.135452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74af21-043f-420d-832c-21bcc23e980b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.168145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.168930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.169215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.169491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.179697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.181141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:28.198389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:28.199479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:28.200621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:28.200621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:28.200621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:28.200621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:28.200621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 643/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:28.205472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:28.306326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0a14-d859-4089-8873-57e4ac5ac735 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.339292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.340446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.340728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.341002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.341726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:28.342067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.343519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.355629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 627/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:28.361455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:28.391787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:28.455671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2adf9b-dcfa-44ce-a035-f8897fc3fae4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.488931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.489706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.489983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.490285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.496686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.498143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:28.511611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:28.512645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:28.513755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 644/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:28.519422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:28.577212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:28.617824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747f42f-470c-4982-a81f-73f60d8cf915 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.649868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.650498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.650781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.651056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.651954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:28.652788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.654730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83b20c-d97d-4a97-877b-788b30999858 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.667239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 628/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:28.673236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:28.764630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:28.773185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5b25a-1b4b-4bc1-9bbe-1999268ef19d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:28.807191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:28.808173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:28.808453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:28.808725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] PATH_INFO: `/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/01927a09fd62479a9640dccdda24f762/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:28.810690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a53c00eca8a494bba1641d08ffd89d1, user_id=01927a09fd62479a9640dccdda24f762, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:28.812141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa72c5b-ad80-4178-89af-3421271e2193 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:28.933548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 645/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 20:08:28 2022] PUT /identity/v3/projects/8a53c00eca8a494bba1641d08ffd89d1/users/01927a09fd62479a9640dccdda24f762/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:08:31.100454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b2b5-b7c5-4714-8a3d-0e3af6a163fe None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.100990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b2b5-b7c5-4714-8a3d-0e3af6a163fe None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.100990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1b2b5-b7c5-4714-8a3d-0e3af6a163fe None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.102393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 629/1274] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:31.108452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.111186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.111186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.125406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:31.126607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:31.190545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 646/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:31.213651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1b2b5-b7c5-4714-8a3d-0e3af6a163fe None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:31.316966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1b2b5-b7c5-4714-8a3d-0e3af6a163fe None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:31.334132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:31.349984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:31.350851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.351083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.351359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.361100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 20:08:31.368745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:31.369842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:31.374450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 630/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 20:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:31.376593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:31.454971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:31.472148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751e8b5-7096-4d84-90ed-56d54cee8fb2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:31.504597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:31.505301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.505582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.505850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.506676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:31.506976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:31.508339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:31.516881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 647/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 20:08:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:31.522378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:31.619748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ebd56-500b-4ab9-929f-38ee29468d62 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:31.653454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:31.654282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.654563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.654834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.665111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:31.666583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:31.683161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:31.684228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:31.685518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:31.685518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:31.685518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:31.685518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:31.685518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 631/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:31.690090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:31.787445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129daaba-8f0e-4ead-9fe6-f946c1ceed8e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:31.818802 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:31.819452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.819734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.820009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.820720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:31.821025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:31.822418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:31.834659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 648/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:31.840777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:31.870384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:31.938319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430b95-0299-4f5b-beb7-ebd138f6da5d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:31.971821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:31.972615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:31.973142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:31.973419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:31.980206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:31.981638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 20:08:31.996602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:32.002355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:32.003443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 632/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:32.004147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.007686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:32.100358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8216459-b89e-4ac7-8a31-dcee99a9981b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:32.133759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:32.134456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:32.134738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:32.135011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:32.135838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:32.136180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:32.137766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:32.154951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 649/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:08:32.160953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.250631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:32.259675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0569b-b1ce-4707-91e0-39550f44ce8b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:32.291886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:32.293126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:32.293410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:32.293683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:32.358946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:32.360374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:32.444415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:32.454578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 633/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 20:08:32.460329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.519406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:32.556649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f093-aa0f-4c78-97d3-8c4c9d5a4c8e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:32.589864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:32.590760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:32.591007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:32.591273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:32.601611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:32.603564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 20:08:32.619656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:32.620735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:32.621856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 20:08:32.621856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 20:08:32.621856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 20:08:32.621856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:32.621856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 650/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:32.626478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.725230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ff915-9b01-471e-95b6-d5a0e752a5a1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:32.757992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:32.758674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:32.758958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:32.759234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:32.759957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:32.760266 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:32.762285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:32.773323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 634/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:32.779134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.877024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b35-2d9e-43fb-9b62-155402469422 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:32.890736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:32.909544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:32.910355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:32.910676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:32.910943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:32.917263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:32.918754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 20:08:32.932593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:32.934064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 23 20:08:32.934969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 651/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:32.940331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:32.990658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:33.039200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1d372-cf3b-4cea-8af5-2baee0d46c2e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:33.071668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:33.072314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:33.072597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:33.072868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:33.074184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:33.074494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:33.075888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31b63-1ed9-4e79-8564-10ce0a5f8895 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:33.087456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 635/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 20:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:33.094209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:33.191830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47babf18-13a0-42c3-8815-1243cb9b384c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:33.225976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:33.226669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:33.226946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:33.227215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:33.228037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] RBAC: Adding query filter params (role.id=e9cc8b774de34cf99d5bb6930f6ce1c4, scope.project.id=8a53c00eca8a494bba1641d08ffd89d1, user.id=01927a09fd62479a9640dccdda24f762) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:33.228341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:33.229761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e8c8c-dbf6-4ccd-877c-320f9c47e2bd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:33.235503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 652/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 20:08:33 2022] GET /identity/v3/role_assignments?user.id=01927a09fd62479a9640dccdda24f762&scope.project.id=8a53c00eca8a494bba1641d08ffd89d1&role.id=e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:35.471558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084a0-931e-478c-9956-b7c44d9d118b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:35.472350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084a0-931e-478c-9956-b7c44d9d118b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:35.472350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084a0-931e-478c-9956-b7c44d9d118b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:35.473937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 636/1288] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:35.479576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:35.480017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:35.480290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:35.498098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:35.499286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:35.529156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:35.564440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 653/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:35.586733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd084a0-931e-478c-9956-b7c44d9d118b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:35.684714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd084a0-931e-478c-9956-b7c44d9d118b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:35.695008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:35.717468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:35.718283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:35.718563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:35.718835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:35.719480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:35.720821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:35.733252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:35.737771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:35.738946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 637/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:08:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:35.740486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:35.835277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95426778-eaf4-43ae-b462-00ef7e48a2b2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:35.868759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:35.869409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:35.869689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:35.869962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:35.870691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:35.870995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:35.872364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5c2f-07bd-4458-afa9-9d9599da445e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:35.877179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 654/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 20:08:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:35.883016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:35.977813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f5434-92c1-447a-95cd-3564e78b24e6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:36.009703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:36.010313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:36.010596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:36.010869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:36.011611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] RBAC: Adding query filter params (type=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:36.011917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:36.013270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d42dd0-59d5-4f31-9b87-0902077146bd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:36.017535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 638/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 20:08:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:08:38.203335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f95-6414-4b43-b15a-9089a5294749 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:38.203886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f95-6414-4b43-b15a-9089a5294749 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:38.203999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c680f95-6414-4b43-b15a-9089a5294749 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:38.205286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 655/1293] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:38.210408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07d69b-f45d-4b66-bdf0-71a0f578acb0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:38.212490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07d69b-f45d-4b66-bdf0-71a0f578acb0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:38.212490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07d69b-f45d-4b66-bdf0-71a0f578acb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:38.228287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3c07d69b-f45d-4b66-bdf0-71a0f578acb0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:38.229481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3c07d69b-f45d-4b66-bdf0-71a0f578acb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:38.258511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:38.294777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 639/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:38.317626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c680f95-6414-4b43-b15a-9089a5294749 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:38.348211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:38.416103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c680f95-6414-4b43-b15a-9089a5294749 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:38.450230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:38.450883 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:38.451167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:38.451444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:38.452157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:38.453584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdbee5-80d1-4662-b6be-a837b2e3f731 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:38.597927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 656/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 20:08:38 2022] POST /identity/v3/services => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:08:40.820562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8826e-2b3c-443e-86f3-3d586ff8b167 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:40.821105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8826e-2b3c-443e-86f3-3d586ff8b167 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:40.821105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8826e-2b3c-443e-86f3-3d586ff8b167 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:40.822537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 640/1296] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:40.828209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:40.829757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:40.829757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:40.845922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:40.847129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:40.918284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 657/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:40.941201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8826e-2b3c-443e-86f3-3d586ff8b167 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:41.038693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8826e-2b3c-443e-86f3-3d586ff8b167 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:41.070756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:41.071593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:41.071870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:41.072140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:41.072781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:41.074744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:41.086579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:41.087763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:41.088657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:41.090163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 641/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:08:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:41.093938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:41.112105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:41.191544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a46d3-2106-4290-9503-c842338b3d0b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:41.225848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:41.226542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:41.226825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:41.227099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:41.227843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:41.228154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:41.229723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:41.235437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 658/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 20:08:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:41.241662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:41.272050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:41.337791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4b895-3804-46d4-b106-88de8186b3b8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:41.371923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:41.372572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:41.372856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:41.373132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:41.373920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] RBAC: Adding query filter params (interface=public, service_id=e76111e43b3b49e795646ad18d1707e3, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:41.374264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:41.375624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3f264-8d8f-4139-959a-6e93c29adc9e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:41.380606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 642/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 20:08:41 2022] GET /identity/v3/endpoints?service_id=e76111e43b3b49e795646ad18d1707e3&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:41.386228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:41.465328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:41.483045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f1e4f-a0f6-414a-ae53-3dadf208630b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:41.515194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:41.515840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:41.516117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:41.516384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:41.517051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:41.518485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e795d-6b38-417f-96e3-a95b1739b1fa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:41.525165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 659/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 20:08:41 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:08:43.720347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f540-4a62-410a-906d-58e634b2352b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:43.720903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f540-4a62-410a-906d-58e634b2352b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:43.720903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f540-4a62-410a-906d-58e634b2352b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:43.722305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 643/1302] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:08:43.727881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:43.728313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:43.728456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:43.745281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:08:43.746497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:08:43.810453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 660/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:08:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 20:08:43.832851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477f540-4a62-410a-906d-58e634b2352b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:43.924656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:43.931109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477f540-4a62-410a-906d-58e634b2352b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:43.963601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:43.964387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:43.965235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:43.965507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:43.966186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:43.967541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 20:08:43.979659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 20:08:43.980859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 23 20:08:43.981979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 644/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 20:08:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 20:08:43.987094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:44.061035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:44.082610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf9097-7c9e-46af-89b1-9df62a33cb96 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:44.115909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:44.116669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:44.117175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:44.117636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:44.118431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:08:44.119130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:44.120497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d981-69ef-4b9b-aef7-403e7631dfcd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:44.126364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 661/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 20:08:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:08:44.132212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:08:44.228611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d41512-8738-460d-a302-d6502bc95447 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:08:44.262357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:08:44.262928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:08:44.263211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:08:44.263486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:08:44.264220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:08:44.265565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9606cb-c1c9-47cc-9334-016568cf83c4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:08:44.334589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 645/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 20:08:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:08:54.123186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=30 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:08:54.330103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:21.508925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3845c-2874-4ab4-b07b-c0398a69625d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:21.510183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3845c-2874-4ab4-b07b-c0398a69625d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:21.510689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3845c-2874-4ab4-b07b-c0398a69625d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:21.513211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 662/1307] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:21.518415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edddd2c-000f-437d-863a-3349f30751b1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:21.518859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edddd2c-000f-437d-863a-3349f30751b1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:21.519082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edddd2c-000f-437d-863a-3349f30751b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:21.606956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3edddd2c-000f-437d-863a-3349f30751b1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:21.608181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3edddd2c-000f-437d-863a-3349f30751b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:21.776375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 646/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:21.831633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:21.831633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:21.831633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:21.831633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 663/1309] 10.222.0.16 () {54 vars in 964 bytes} [Mon May 23 20:14:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:21.836007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e21c1-9fbd-4c38-8ae5-ce0c57451f3f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:21.836402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e21c1-9fbd-4c38-8ae5-ce0c57451f3f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:21.836872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e21c1-9fbd-4c38-8ae5-ce0c57451f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:21.938336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e82e21c1-9fbd-4c38-8ae5-ce0c57451f3f None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:21.943447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e82e21c1-9fbd-4c38-8ae5-ce0c57451f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:21.974240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:22.046326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 647/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Mon May 23 20:14:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:22.051491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:22.053387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:22.163365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512ecf6-d8a6-4a0a-a628-27060bc85fca None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:22.242160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:22.242812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:22.243095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:22.243372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:22.245005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:22.299009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:22.354495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:22.355975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064a10b-02e7-400a-821b-fd73b4b8e956 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:22.375230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 664/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:24.869777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:24.870344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:24.870344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:24.872619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 648/1312] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:24.878082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31525-2814-4c81-abdd-20aa79646dc0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:24.878560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31525-2814-4c81-abdd-20aa79646dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:24.878683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31525-2814-4c81-abdd-20aa79646dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:24.956297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1ac31525-2814-4c81-abdd-20aa79646dc0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:24.957656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1ac31525-2814-4c81-abdd-20aa79646dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:25.078060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 665/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:25.116343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:25.117870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:25.228424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8553-d102-417f-bf5a-921921872bd3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:25.302633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:25.305249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:25.305903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:25.306207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:25.306480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:25.308196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:25.417811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:25.419297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd39e-eea6-4383-ab52-5a1a72367546 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:25.437680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 649/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:28.121422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:28.121974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:28.121974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:28.123355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 666/1315] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:28.128677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e091-30f8-42b1-aad9-35dab8e06e66 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:28.129163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e091-30f8-42b1-aad9-35dab8e06e66 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:28.129269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e091-30f8-42b1-aad9-35dab8e06e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:28.191853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ded5e091-30f8-42b1-aad9-35dab8e06e66 None None] MFA Rules not processed for user `15a2f040e2f14226954b7f7b777d1ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:28.193001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ded5e091-30f8-42b1-aad9-35dab8e06e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:28.244167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:28.287401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 650/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:14:28.326056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:28.327588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:28.337148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:28.430542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807abb75-d717-4135-ab3e-2ad02d7f63b5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:28.501305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:28.501884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:28.502202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:28.502480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:28.504148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:28.564096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:28.610459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:28.611906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85cb4e-6983-4816-9faf-43d59032a6c1 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:28.629415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 667/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:31.532148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:31.532666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:31.532666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:31.534111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 651/1318] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:31.539349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca72-ed19-416b-9fed-cbdd32c418c9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:31.539745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca72-ed19-416b-9fed-cbdd32c418c9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:31.539899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca72-ed19-416b-9fed-cbdd32c418c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:31.606653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e287ca72-ed19-416b-9fed-cbdd32c418c9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:31.607869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e287ca72-ed19-416b-9fed-cbdd32c418c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:31.703386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 668/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:31.741714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:31.743274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:31.843462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7627bd-6978-4461-adff-d3958d3dd8aa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:31.914000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:31.914673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:31.914957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:31.915229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:31.916911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:31.924087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:32.022060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:32.023490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fb24-a501-4061-9528-f407f62f0c5d service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:32.040705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 652/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:35.195049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:35.195617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:35.195617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:35.196970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 669/1321] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:35.202911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07ea0-586b-4542-a3e0-11728c722d49 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:35.203315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07ea0-586b-4542-a3e0-11728c722d49 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:35.203505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07ea0-586b-4542-a3e0-11728c722d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:35.270044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0ca07ea0-586b-4542-a3e0-11728c722d49 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:35.271230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0ca07ea0-586b-4542-a3e0-11728c722d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:35.328045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:35.366520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 653/1322] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:14:35.402540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:35.404146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:35.423859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:35.510695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b187bdb-d59f-4d8c-96f6-fc930ae4e4d5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:35.586313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:35.587206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:35.587625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:35.588037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:35.590187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:35.661368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:35.697537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:35.699005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30a3f-f884-438f-a107-f0112cf4b47c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:35.717757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 670/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:37.955043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:37.955598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:37.955598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:37.957092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 654/1324] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:37.963179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944944c-2b95-451d-83b7-17d21bdb3647 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:37.963733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944944c-2b95-451d-83b7-17d21bdb3647 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:37.963733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944944c-2b95-451d-83b7-17d21bdb3647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:38.028474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a944944c-2b95-451d-83b7-17d21bdb3647 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:38.029676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a944944c-2b95-451d-83b7-17d21bdb3647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:38.123870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 671/1325] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:38.159790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:38.161406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:38.266734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3291fe-7cdf-4de7-9376-b1d404c6308b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:38.337690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:38.338375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:38.338667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:38.338944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:38.340635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:38.357661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:38.446613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:38.448055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2decc1-d5f0-4993-ab0d-b1fd6b9ea6b0 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:38.466985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 655/1326] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:40.710390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc2e0-36b2-454b-9d07-54c8c9b83847 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:40.710947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc2e0-36b2-454b-9d07-54c8c9b83847 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:40.710947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc2e0-36b2-454b-9d07-54c8c9b83847 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:40.712320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 672/1327] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:40.718676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232701-d674-4d14-954a-077387f18203 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:40.719128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232701-d674-4d14-954a-077387f18203 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:40.719239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232701-d674-4d14-954a-077387f18203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:40.906149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e3232701-d674-4d14-954a-077387f18203 None None] MFA Rules not processed for user `15a2f040e2f14226954b7f7b777d1ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:40.907350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e3232701-d674-4d14-954a-077387f18203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:40.972988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:41.005391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 656/1328] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:41.055555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:41.055845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:41.056122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:41.057535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 673/1329] 10.222.0.16 () {54 vars in 964 bytes} [Mon May 23 20:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:41.063576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acfd7-27e4-48e2-a61e-6ec1034f5192 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:41.063863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acfd7-27e4-48e2-a61e-6ec1034f5192 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:41.064136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acfd7-27e4-48e2-a61e-6ec1034f5192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:41.136468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e18acfd7-27e4-48e2-a61e-6ec1034f5192 None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:41.138160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e18acfd7-27e4-48e2-a61e-6ec1034f5192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:41.211993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 657/1330] 10.222.0.16 () {58 vars in 1061 bytes} [Mon May 23 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:41.222140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:41.223930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:41.253786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:41.325251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540cef-8eb6-4905-9095-725b181d6efe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:41.396336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:41.396980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:41.397262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:41.397536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:41.399209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:41.475744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:41.507264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:41.508709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0c9b1-4fc2-49aa-8c34-9d89ada69a10 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:41.528131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 674/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:44.549723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:44.550348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:44.550348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:44.551688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 658/1332] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:44.556982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd825a-aded-4e3b-969a-e231794e53ad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:44.557420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd825a-aded-4e3b-969a-e231794e53ad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:44.557527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd825a-aded-4e3b-969a-e231794e53ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:44.621560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-98cd825a-aded-4e3b-969a-e231794e53ad None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:44.622868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-98cd825a-aded-4e3b-969a-e231794e53ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:44.719054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 675/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:44.754214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:44.755717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:44.823467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:44.858552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1436b0-ea23-4f97-960f-7c50c24ed156 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:44.929753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:44.930437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:44.930722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:44.931027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:44.932658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:45.039967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:45.041352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3bfab-75c3-4be3-9479-52ad2b9ff054 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:45.051271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:45.059899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 659/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:47.305939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:47.306538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:47.306538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:47.307881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 676/1335] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:47.313424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4693807-f37b-42c1-9161-9f970205e512 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:47.313844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4693807-f37b-42c1-9161-9f970205e512 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:47.313979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4693807-f37b-42c1-9161-9f970205e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:47.376846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f4693807-f37b-42c1-9161-9f970205e512 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:47.378060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f4693807-f37b-42c1-9161-9f970205e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:47.474479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 660/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:47.513219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:47.514788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:47.554164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:47.617795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9df4c2-f030-4829-8039-35b13f2c6df6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:47.692270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:47.692921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:47.693203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:47.693478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:47.695157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:47.778934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:47.803840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:47.805404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5cf33-3a15-4d5c-80e9-352dba12e51a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:47.825887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 677/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:50.515286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:50.515804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:50.515804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:50.517099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 661/1338] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:50.522232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7a82-b023-43f2-810c-b2251bda5fb7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:50.522657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7a82-b023-43f2-810c-b2251bda5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:50.522794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7a82-b023-43f2-810c-b2251bda5fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:50.586753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6fbd7a82-b023-43f2-810c-b2251bda5fb7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:50.587994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6fbd7a82-b023-43f2-810c-b2251bda5fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:50.682776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 678/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:50.717459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:50.718995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:50.793185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:50.819864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8559f0-e978-4c66-95d6-83d93856a4a6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:50.894293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:50.894942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:50.895222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:50.895491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:50.897097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:50.999892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:51.001302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f9dc-83e5-442d-8627-a555d2ca5239 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:51.019399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 662/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:55.532332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:55.532860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:55.532860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:55.534430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 679/1341] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:55.539582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927eadc-b794-4696-a867-448757347c7f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:55.540014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927eadc-b794-4696-a867-448757347c7f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:55.540136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927eadc-b794-4696-a867-448757347c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:55.547601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:55.624132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5927eadc-b794-4696-a867-448757347c7f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:55.625500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5927eadc-b794-4696-a867-448757347c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:55.731444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 663/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:55.771499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:55.773647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:55.823045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:14:55.869143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef28e-29ec-4def-b9ef-828c06246e5c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:55.872769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:55.873431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:55.873709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:55.873980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:55.882852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:55.884211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983bdd5-788a-4526-90d9-c67fb264f69b service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:55.893521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 680/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:14:59.131252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebd3e-ebc4-4c1c-b43f-9f6ebe9e5354 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:59.131794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebd3e-ebc4-4c1c-b43f-9f6ebe9e5354 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:59.131794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebd3e-ebc4-4c1c-b43f-9f6ebe9e5354 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:59.133258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 664/1344] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:14:59.139501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff756b-1181-4063-9f01-cd9125d0e48e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:59.139932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff756b-1181-4063-9f01-cd9125d0e48e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:59.140066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff756b-1181-4063-9f01-cd9125d0e48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:59.168252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-79ff756b-1181-4063-9f01-cd9125d0e48e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:14:59.169440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-79ff756b-1181-4063-9f01-cd9125d0e48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:14:59.248751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 681/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:14:59.284829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ebd3e-ebc4-4c1c-b43f-9f6ebe9e5354 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:14:59.296010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ebd3e-ebc4-4c1c-b43f-9f6ebe9e5354 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:14:59.299504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:14:59.300143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:14:59.300426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:14:59.300732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:14:59.310552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:14:59.311973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5d807-b7be-48e3-ba32-454b6044e563 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:14:59.322594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 665/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:03.871710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:03.872220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:03.872471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:03.873847 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 682/1347] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:03.879368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60760fda-769a-4904-8956-a0c937299b79 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:03.880641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60760fda-769a-4904-8956-a0c937299b79 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:03.880641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60760fda-769a-4904-8956-a0c937299b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:03.898351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-60760fda-769a-4904-8956-a0c937299b79 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:03.899661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60760fda-769a-4904-8956-a0c937299b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:03.908355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 666/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:03.946333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:03.948590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:03.961292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e961889-db13-423f-9d5d-5fb8a66f4ec6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:03.966815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:03.966815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:03.966815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:03.966815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:03.975591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:03.977020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88381-7724-4ed4-8964-f5e7fc605baa service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:03.986272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 683/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:09.165492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6da9c-9e02-4327-ac50-1a2cc7f00e43 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:09.166405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6da9c-9e02-4327-ac50-1a2cc7f00e43 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:09.166722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6da9c-9e02-4327-ac50-1a2cc7f00e43 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:09.172243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 667/1350] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:09.173740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cd32-3bd5-4fd7-b9ba-cf51ca049d4c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:09.173740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cd32-3bd5-4fd7-b9ba-cf51ca049d4c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:09.174257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451cd32-3bd5-4fd7-b9ba-cf51ca049d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:09.282041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b451cd32-3bd5-4fd7-b9ba-cf51ca049d4c None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:09.283609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b451cd32-3bd5-4fd7-b9ba-cf51ca049d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:09.354422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:09.418803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 684/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:15:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:13.893241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=72 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:14.021903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36099-2110-4f08-88a3-4274ccb652a7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:14.022627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36099-2110-4f08-88a3-4274ccb652a7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:14.023023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e36099-2110-4f08-88a3-4274ccb652a7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:14.024521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 668/1352] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:14.030342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f350-99e6-40d2-9b54-1fb481780ed3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:14.030787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f350-99e6-40d2-9b54-1fb481780ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:14.031081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f350-99e6-40d2-9b54-1fb481780ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:14.050042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cd62f350-99e6-40d2-9b54-1fb481780ed3 None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:14.051428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cd62f350-99e6-40d2-9b54-1fb481780ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:14.061159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 685/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:15:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:15.279478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3d1e-1f13-436e-b509-4fad507555d4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:15.280000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3d1e-1f13-436e-b509-4fad507555d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:15.280146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e3d1e-1f13-436e-b509-4fad507555d4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:15.281636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 669/1354] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:15.286556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff45f-e499-460e-8b89-3ebb1d42221b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:15.286992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff45f-e499-460e-8b89-3ebb1d42221b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:15.287115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff45f-e499-460e-8b89-3ebb1d42221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:15.305327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-439ff45f-e499-460e-8b89-3ebb1d42221b None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:15.306794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-439ff45f-e499-460e-8b89-3ebb1d42221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:15.316335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 686/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:19.129853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574900-1fd4-4d19-8714-bba10e89f459 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:19.130430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574900-1fd4-4d19-8714-bba10e89f459 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:19.130430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574900-1fd4-4d19-8714-bba10e89f459 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:19.131782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 670/1356] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:19.137128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be466966-41c0-45c0-8c00-16570c9d2d6c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:19.137545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be466966-41c0-45c0-8c00-16570c9d2d6c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:19.137668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be466966-41c0-45c0-8c00-16570c9d2d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:19.156289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-be466966-41c0-45c0-8c00-16570c9d2d6c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:19.157467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-be466966-41c0-45c0-8c00-16570c9d2d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:19.165273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 687/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:19.226863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fcbf-b945-44ca-8cf5-087f1be75702 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:19.227332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fcbf-b945-44ca-8cf5-087f1be75702 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:19.227906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fcbf-b945-44ca-8cf5-087f1be75702 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:19.229364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 671/1358] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 20:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:19.239413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899f44-e106-4ba4-8db1-6ced9292d550 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:19.239966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899f44-e106-4ba4-8db1-6ced9292d550 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:19.240487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899f44-e106-4ba4-8db1-6ced9292d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:19.295680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f8899f44-e106-4ba4-8db1-6ced9292d550 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:19.297531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f8899f44-e106-4ba4-8db1-6ced9292d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:19.391519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 688/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:15:19.399166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112fcbf-b945-44ca-8cf5-087f1be75702 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:19.415133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112fcbf-b945-44ca-8cf5-087f1be75702 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:19.421683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:19.422360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:19.422648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:19.422923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:19.433074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:19.434699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223beb82-f958-4b30-baf7-57860ee58ec5 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:19.446755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 672/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:21.873929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6dba-2d26-4a2a-a8ce-e85c90d29529 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:21.874505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6dba-2d26-4a2a-a8ce-e85c90d29529 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:21.874505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6dba-2d26-4a2a-a8ce-e85c90d29529 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:21.876022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 689/1361] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:21.881156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73b41-68c0-4ec4-b75d-8622e575543d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:21.881565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73b41-68c0-4ec4-b75d-8622e575543d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:21.881704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73b41-68c0-4ec4-b75d-8622e575543d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:21.899664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ada73b41-68c0-4ec4-b75d-8622e575543d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:21.901531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ada73b41-68c0-4ec4-b75d-8622e575543d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:21.909743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 673/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:21.955792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c6dba-2d26-4a2a-a8ce-e85c90d29529 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:21.971167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c6dba-2d26-4a2a-a8ce-e85c90d29529 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:21.976646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:21.977355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:21.977705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:21.978075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:21.986770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:21.988537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22679b3c-a8b9-4350-9d0a-36413f7878dc service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:21.999376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 690/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:24.659868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d710-ab1e-4b5b-b547-fcd8744bdfae None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:24.660324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d710-ab1e-4b5b-b547-fcd8744bdfae None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:24.660421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628d710-ab1e-4b5b-b547-fcd8744bdfae None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:24.661822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 674/1364] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:24.667339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9768df-bdfc-4936-9971-8faa38882ff2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:24.667763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9768df-bdfc-4936-9971-8faa38882ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:24.667900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9768df-bdfc-4936-9971-8faa38882ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:24.686880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5f9768df-bdfc-4936-9971-8faa38882ff2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:24.688088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5f9768df-bdfc-4936-9971-8faa38882ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:24.695912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 691/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:24.741260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1196b-cca8-4076-9003-22040468ab2a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:24.741671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1196b-cca8-4076-9003-22040468ab2a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:24.742056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1196b-cca8-4076-9003-22040468ab2a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:24.750433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 675/1366] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 20:15:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:24.755476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a123-1ec8-4493-b749-085f19faab94 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:24.755781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a123-1ec8-4493-b749-085f19faab94 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:24.756055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a123-1ec8-4493-b749-085f19faab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:24.782774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3878a123-1ec8-4493-b749-085f19faab94 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:24.783899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3878a123-1ec8-4493-b749-085f19faab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:24.793350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 692/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:24.801395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1196b-cca8-4076-9003-22040468ab2a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:24.815870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1196b-cca8-4076-9003-22040468ab2a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:24.821227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:24.821864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:24.822191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:24.822981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:24.830860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:24.832434 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078f1de-e543-4deb-aac3-c97564a2d1a2 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:24.842799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 676/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:27.295029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a033-43cf-4470-be41-9a9c48bc3a14 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.295592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a033-43cf-4470-be41-9a9c48bc3a14 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.295592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2a033-43cf-4470-be41-9a9c48bc3a14 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.297205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 693/1369] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:27.302823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387cca8-4579-4b71-998e-1f906b96a4de None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.303288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387cca8-4579-4b71-998e-1f906b96a4de None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.303365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387cca8-4579-4b71-998e-1f906b96a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.321078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0387cca8-4579-4b71-998e-1f906b96a4de None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:27.322473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0387cca8-4579-4b71-998e-1f906b96a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:27.330874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 677/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:27.375851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2a033-43cf-4470-be41-9a9c48bc3a14 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:27.390273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2a033-43cf-4470-be41-9a9c48bc3a14 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:27.395719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:27.396364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.396650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.396929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.406075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:27.407687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787c330-17de-4f21-a99e-a781de2630d7 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:27.418342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 694/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:27.466299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735c14-61d4-4e60-a3ab-9a35718e8a5c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.466831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735c14-61d4-4e60-a3ab-9a35718e8a5c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.467186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8735c14-61d4-4e60-a3ab-9a35718e8a5c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.468722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 678/1372] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:27.478212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62803c7-29e3-40ba-9547-5b10014e14aa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.478212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62803c7-29e3-40ba-9547-5b10014e14aa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.478212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62803c7-29e3-40ba-9547-5b10014e14aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.495125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e62803c7-29e3-40ba-9547-5b10014e14aa None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:27.496362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e62803c7-29e3-40ba-9547-5b10014e14aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:27.506200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 695/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:15:27.855259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7e7b-c545-4499-bcef-4c6d7467f850 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.855259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7e7b-c545-4499-bcef-4c6d7467f850 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.855259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7e7b-c545-4499-bcef-4c6d7467f850 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.859013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 679/1374] 10.222.0.16 () {54 vars in 963 bytes} [Mon May 23 20:15:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:27.865701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcb6c8-9a9c-4c9b-b0b7-63b73b07cff9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:27.866008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcb6c8-9a9c-4c9b-b0b7-63b73b07cff9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:27.866318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcb6c8-9a9c-4c9b-b0b7-63b73b07cff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:27.874889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:27.917732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d3fcb6c8-9a9c-4c9b-b0b7-63b73b07cff9 None None] MFA Rules not processed for user `567602cb67c447c798d4009f77140b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:27.919014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3fcb6c8-9a9c-4c9b-b0b7-63b73b07cff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:27.984651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 696/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Mon May 23 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:27.992553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b7e7b-c545-4499-bcef-4c6d7467f850 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:28.005295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b7e7b-c545-4499-bcef-4c6d7467f850 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:28.009021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:28.009692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:28.009983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:28.010599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:28.021316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:28.022941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626922b-4150-4172-88a3-c9afcfc0dcd3 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:28.034051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 680/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:29.764341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88b11d-6fa0-4430-9164-90bc3d7ac0d8 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:29.764896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88b11d-6fa0-4430-9164-90bc3d7ac0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:29.764896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88b11d-6fa0-4430-9164-90bc3d7ac0d8 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:29.766563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 697/1377] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:29.772407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697bcbd-050d-4c1d-bf8e-7c5d1a6edf12 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:29.772982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697bcbd-050d-4c1d-bf8e-7c5d1a6edf12 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:29.773074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697bcbd-050d-4c1d-bf8e-7c5d1a6edf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:29.794506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7697bcbd-050d-4c1d-bf8e-7c5d1a6edf12 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:29.797922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7697bcbd-050d-4c1d-bf8e-7c5d1a6edf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:29.804093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 681/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:29.849147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88b11d-6fa0-4430-9164-90bc3d7ac0d8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:29.864741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88b11d-6fa0-4430-9164-90bc3d7ac0d8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:29.870501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:29.871166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:29.871449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:29.871723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:29.880846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:29.882515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b133-0c45-4828-a5ab-e97725b9e529 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:29.893646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 698/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:32.251281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126baba8-c2db-4876-a105-ad668cf7b03e None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:32.251812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126baba8-c2db-4876-a105-ad668cf7b03e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:32.251812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126baba8-c2db-4876-a105-ad668cf7b03e None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:32.253193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 682/1380] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:32.258635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdda57-2b27-4240-88e6-5efc7766807e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:32.259084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdda57-2b27-4240-88e6-5efc7766807e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:32.259181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdda57-2b27-4240-88e6-5efc7766807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:32.277100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-44cdda57-2b27-4240-88e6-5efc7766807e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:32.278325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-44cdda57-2b27-4240-88e6-5efc7766807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:32.286539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 699/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:32.330791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126baba8-c2db-4876-a105-ad668cf7b03e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:32.345454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126baba8-c2db-4876-a105-ad668cf7b03e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:32.351159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:32.351802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:32.352084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:32.352375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:32.361186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:32.362789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7026e-ae62-4425-8c43-449e946c0495 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:32.373252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 683/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:34.694412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec86e-237a-42ed-8e2f-a255bb0faf1f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:34.695001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec86e-237a-42ed-8e2f-a255bb0faf1f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:34.695001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec86e-237a-42ed-8e2f-a255bb0faf1f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:34.696458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 700/1383] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:34.702814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdc22b-e804-4f4e-a32e-17f547367841 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:34.703413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdc22b-e804-4f4e-a32e-17f547367841 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:34.703413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdc22b-e804-4f4e-a32e-17f547367841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:34.723279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e7fdc22b-e804-4f4e-a32e-17f547367841 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:34.724522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e7fdc22b-e804-4f4e-a32e-17f547367841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:34.733202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 684/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:34.774435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bec86e-237a-42ed-8e2f-a255bb0faf1f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:34.790551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bec86e-237a-42ed-8e2f-a255bb0faf1f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:34.796842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:34.797666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:34.798135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:34.798552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:34.807457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:34.809218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee5037-2c22-42e8-9fad-d00b3e0e5a09 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:34.819255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 701/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:37.008405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c14e-9d34-4a6c-a0ca-073697b53cb7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:37.009139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c14e-9d34-4a6c-a0ca-073697b53cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:37.009139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c14e-9d34-4a6c-a0ca-073697b53cb7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:37.010346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 685/1386] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:37.016186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c0599-60a1-4926-acb3-66a192bc6677 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:37.016623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c0599-60a1-4926-acb3-66a192bc6677 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:37.016922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c0599-60a1-4926-acb3-66a192bc6677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:37.040349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e61c0599-60a1-4926-acb3-66a192bc6677 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:37.042101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e61c0599-60a1-4926-acb3-66a192bc6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:37.052500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 702/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:37.096155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646c14e-9d34-4a6c-a0ca-073697b53cb7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:37.110512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646c14e-9d34-4a6c-a0ca-073697b53cb7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:37.115929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:37.116566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:37.116847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:37.117118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:37.126133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:37.127705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07517d6-230e-40b8-a680-886118d99a36 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:37.138416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 686/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:39.478657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deba2-1165-41b8-b165-35d16031fed3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:39.479714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deba2-1165-41b8-b165-35d16031fed3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:39.479983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deba2-1165-41b8-b165-35d16031fed3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:39.481889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 703/1389] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:39.487495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca372-e1dd-4203-994b-676a731b78e2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:39.487989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca372-e1dd-4203-994b-676a731b78e2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:39.488144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ca372-e1dd-4203-994b-676a731b78e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:39.505314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-923ca372-e1dd-4203-994b-676a731b78e2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:39.506556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-923ca372-e1dd-4203-994b-676a731b78e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:39.514658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 687/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:39.569353 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2deba2-1165-41b8-b165-35d16031fed3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:39.584688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2deba2-1165-41b8-b165-35d16031fed3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:39.590701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:39.591692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:39.591692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:39.591934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:39.600597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:39.602222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c7c2e-d74b-48c6-bb98-85741afcae71 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:39.612029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 704/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:42.427229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed481-c168-4aa4-86fb-37578d115afb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:42.427837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed481-c168-4aa4-86fb-37578d115afb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:42.427837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed481-c168-4aa4-86fb-37578d115afb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:42.429183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 688/1392] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:42.435093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a7dc-1617-46b6-aec5-b7543e5ea389 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:42.435544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a7dc-1617-46b6-aec5-b7543e5ea389 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:42.435653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a7dc-1617-46b6-aec5-b7543e5ea389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:42.453899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ea17a7dc-1617-46b6-aec5-b7543e5ea389 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:42.455136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ea17a7dc-1617-46b6-aec5-b7543e5ea389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:42.463026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 705/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:15:42.507529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed481-c168-4aa4-86fb-37578d115afb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:42.522165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed481-c168-4aa4-86fb-37578d115afb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:42.527467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:42.528099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:42.528383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:42.528653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:42.536525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:42.538106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd30b5-c6f9-407e-be03-0cc25526cf52 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:42.547272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 689/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:43.903445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504908bb-9bce-4a6e-8c43-140db93632fd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:43.904016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504908bb-9bce-4a6e-8c43-140db93632fd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:43.904016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504908bb-9bce-4a6e-8c43-140db93632fd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:43.905382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 706/1395] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:43.910795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff766-0d4d-4770-8c25-d3b1b9abc0f2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:43.911216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff766-0d4d-4770-8c25-d3b1b9abc0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:43.911350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff766-0d4d-4770-8c25-d3b1b9abc0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:43.924579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=54 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:43.930237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-106ff766-0d4d-4770-8c25-d3b1b9abc0f2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:43.931422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-106ff766-0d4d-4770-8c25-d3b1b9abc0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:43.939339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 690/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:43.998225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733ca4-232f-4b0d-9f55-ea688f7247d5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:43.999077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733ca4-232f-4b0d-9f55-ea688f7247d5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:43.999427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733ca4-232f-4b0d-9f55-ea688f7247d5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.000913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 707/1397] 10.222.0.16 () {54 vars in 962 bytes} [Mon May 23 20:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:44.006473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc647e5-560b-425f-869f-385c27673ccc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:44.007021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc647e5-560b-425f-869f-385c27673ccc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:44.007307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc647e5-560b-425f-869f-385c27673ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.060159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9fc647e5-560b-425f-869f-385c27673ccc None None] MFA Rules not processed for user `bef73ca7ffe94b79b085dcfa1eae278d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:44.061347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9fc647e5-560b-425f-869f-385c27673ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:44.120397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 691/1398] 10.222.0.16 () {58 vars in 1059 bytes} [Mon May 23 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:44.128258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6733ca4-232f-4b0d-9f55-ea688f7247d5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:44.140572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6733ca4-232f-4b0d-9f55-ea688f7247d5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:44.144107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:44.144746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:44.145032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:44.145311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.155004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:44.156374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0eb5ef-af56-438d-bc68-cd459ee65658 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:44.165590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 708/1399] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:44.862848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c0e3-cdec-41ab-8741-a86b3a71904f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:44.863165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c0e3-cdec-41ab-8741-a86b3a71904f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:44.863522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c0e3-cdec-41ab-8741-a86b3a71904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.911475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4592c0e3-cdec-41ab-8741-a86b3a71904f None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:44.912670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4592c0e3-cdec-41ab-8741-a86b3a71904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:44.950153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2aa92-2866-44a9-aa2e-52c7ce731b64 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:44.950685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2aa92-2866-44a9-aa2e-52c7ce731b64 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:44.950685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2aa92-2866-44a9-aa2e-52c7ce731b64 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.952059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 709/1400] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:15:44.958298 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034852-929e-40a7-9855-65afdcef3736 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:44.958647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034852-929e-40a7-9855-65afdcef3736 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:44.959011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034852-929e-40a7-9855-65afdcef3736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:44.977428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-93034852-929e-40a7-9855-65afdcef3736 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:44.977818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 692/1401] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:44.978585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-93034852-929e-40a7-9855-65afdcef3736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:44.986389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 710/1402] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:15:45.021556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49caad-3d10-4a3a-8fc1-d965cee14cc6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:45.021962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49caad-3d10-4a3a-8fc1-d965cee14cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:45.022313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49caad-3d10-4a3a-8fc1-d965cee14cc6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:45.023732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 693/1403] 10.222.0.16 () {54 vars in 942 bytes} [Mon May 23 20:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:45.031205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434b8e-d57f-4079-8d97-3e6112aca51c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:45.031358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49caad-3d10-4a3a-8fc1-d965cee14cc6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:45.031495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434b8e-d57f-4079-8d97-3e6112aca51c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:45.031779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434b8e-d57f-4079-8d97-3e6112aca51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:45.045361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49caad-3d10-4a3a-8fc1-d965cee14cc6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:45.050972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:45.051613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:45.051901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:45.052182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:45.060453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:45.062043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:45.071511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 694/1404] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:15:45.080921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e434b8e-d57f-4079-8d97-3e6112aca51c None None] MFA Rules not processed for user `1cc42255281f442bb146fe386f432cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:45.082166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e434b8e-d57f-4079-8d97-3e6112aca51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:45.115023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:45.163970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 711/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Mon May 23 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:45.173359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:45.184574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112fad-2190-4c09-85d2-66bfc0c1107d service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:45.189747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:45.190419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:45.190692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:45.190971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:45.199682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:45.201257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eaa4d3-7c96-4f77-9d9e-78f7671281da service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:45.209319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 695/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:15:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:15:47.476274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a248-71a1-4753-863a-3c039b79e182 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:47.476787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a248-71a1-4753-863a-3c039b79e182 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:47.476787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793a248-71a1-4753-863a-3c039b79e182 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:47.478134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 712/1407] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:15:47.484386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a748dd-9bb1-4171-b3b5-00ca02469680 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:47.484951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a748dd-9bb1-4171-b3b5-00ca02469680 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:47.484951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a748dd-9bb1-4171-b3b5-00ca02469680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:47.506826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-62a748dd-9bb1-4171-b3b5-00ca02469680 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:15:47.508535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-62a748dd-9bb1-4171-b3b5-00ca02469680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:15:47.522605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 696/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:15:47.582671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793a248-71a1-4753-863a-3c039b79e182 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:15:47.588774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793a248-71a1-4753-863a-3c039b79e182 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:15:47.595047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:15:47.595154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:15:47.595437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:15:47.595721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:15:47.605200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:15:47.606840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f0222-6c91-49ae-9cd3-a1f06f7fac2a service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:15:47.627135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 713/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:15:57.501143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:15:57.612107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=22 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:16:09.589066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b87464-36be-4ebd-bb26-1fd6eeb113ed None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:09.589908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b87464-36be-4ebd-bb26-1fd6eeb113ed None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:09.589908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b87464-36be-4ebd-bb26-1fd6eeb113ed None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:09.591186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 697/1410] 10.222.0.16 () {54 vars in 918 bytes} [Mon May 23 20:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:09.596138 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371892-1ace-4c45-a609-737c95f7eeee None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:09.596658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371892-1ace-4c45-a609-737c95f7eeee None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:09.596768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371892-1ace-4c45-a609-737c95f7eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:09.614467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f3371892-1ace-4c45-a609-737c95f7eeee None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:09.615665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f3371892-1ace-4c45-a609-737c95f7eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:09.624437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 714/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Mon May 23 20:16:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:09.638586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fcaf5-9d58-43fc-a5f8-95125fb60594 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:09.638874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fcaf5-9d58-43fc-a5f8-95125fb60594 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:09.639151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fcaf5-9d58-43fc-a5f8-95125fb60594 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:09.640488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 698/1412] 10.222.0.16 () {54 vars in 918 bytes} [Mon May 23 20:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:09.644732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5b6f-918a-4198-8c5c-128a4f2cff0b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:09.645015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5b6f-918a-4198-8c5c-128a4f2cff0b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:09.645288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5b6f-918a-4198-8c5c-128a4f2cff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:09.662334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-61ba5b6f-918a-4198-8c5c-128a4f2cff0b None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:09.663500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-61ba5b6f-918a-4198-8c5c-128a4f2cff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:09.672765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 715/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Mon May 23 20:16:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:12.313955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3c6a-7d4b-4ced-99c8-9da1357d5111 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:12.314519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3c6a-7d4b-4ced-99c8-9da1357d5111 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:12.314519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3c6a-7d4b-4ced-99c8-9da1357d5111 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:12.315903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:12.321172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247b4d-620b-436b-854b-7f8576d26df1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:12.321618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247b4d-620b-436b-854b-7f8576d26df1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:12.321714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247b4d-620b-436b-854b-7f8576d26df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:12.339625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b8247b4d-620b-436b-854b-7f8576d26df1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:12.340806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b8247b4d-620b-436b-854b-7f8576d26df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:12.348359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:12.391547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f3c6a-7d4b-4ced-99c8-9da1357d5111 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:12.405728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f3c6a-7d4b-4ced-99c8-9da1357d5111 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:12.411019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:12.429014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:12.429014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:12.429014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:12.429014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:12.429014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987fcb9-8e04-426d-b81f-d9e03b8362aa service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:12.432192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 700/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:13.387653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b38d9-467c-4ec6-a3b9-b82e194f1845 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:13.388228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b38d9-467c-4ec6-a3b9-b82e194f1845 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:13.388228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b38d9-467c-4ec6-a3b9-b82e194f1845 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:13.389579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:13.394534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa75f75-dbb3-4310-817a-5b09d307dfc2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:13.394950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa75f75-dbb3-4310-817a-5b09d307dfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:13.395066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa75f75-dbb3-4310-817a-5b09d307dfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:13.412395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4fa75f75-dbb3-4310-817a-5b09d307dfc2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:13.413562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4fa75f75-dbb3-4310-817a-5b09d307dfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:13.421656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:13.588122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b38d9-467c-4ec6-a3b9-b82e194f1845 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:13.602716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b38d9-467c-4ec6-a3b9-b82e194f1845 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:13.608176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:13.608807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:13.609090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:13.609368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:13.617838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:13.619504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5620a9-515f-47a4-b8c2-669ba5663d5b service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:13.629394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 718/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:14.549889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e511e1-6e47-4a40-98bc-983d554fddac None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:14.550457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e511e1-6e47-4a40-98bc-983d554fddac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:14.550457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e511e1-6e47-4a40-98bc-983d554fddac None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:14.551764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:14.556596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad743ef-6951-4c87-97ff-544e997a2a7f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:14.557035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad743ef-6951-4c87-97ff-544e997a2a7f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:14.557142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad743ef-6951-4c87-97ff-544e997a2a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:14.574745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-aad743ef-6951-4c87-97ff-544e997a2a7f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:14.575925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aad743ef-6951-4c87-97ff-544e997a2a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:14.583636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:14.733168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e511e1-6e47-4a40-98bc-983d554fddac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:14.747599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e511e1-6e47-4a40-98bc-983d554fddac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:14.752889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:14.753522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:14.753806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:14.754116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:14.762840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:14.764404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed14447-3879-4450-8168-b18026d420b4 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:14.774282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 703/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:15.684756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1ada-bed9-44dd-aabf-82d4a24b7169 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:15.685246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1ada-bed9-44dd-aabf-82d4a24b7169 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:15.685341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b1ada-bed9-44dd-aabf-82d4a24b7169 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:15.686761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:15.691748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44e777-c33d-4e7b-b12c-74c45ace859c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:15.692193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44e777-c33d-4e7b-b12c-74c45ace859c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:15.692311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44e777-c33d-4e7b-b12c-74c45ace859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:15.711249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ce44e777-c33d-4e7b-b12c-74c45ace859c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:15.713024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ce44e777-c33d-4e7b-b12c-74c45ace859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:15.721122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:16:15.868953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b1ada-bed9-44dd-aabf-82d4a24b7169 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:15.883278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b1ada-bed9-44dd-aabf-82d4a24b7169 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:15.889184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:15.889818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:15.890140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:15.890426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:15.898639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:15.900834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7fb0c-482d-45ed-9c8e-cf7f0cb32244 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:15.910493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 721/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:16.808181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3566ff-17cc-483c-bd89-e44b8bcbc9f9 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:16.808759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3566ff-17cc-483c-bd89-e44b8bcbc9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:16.808759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3566ff-17cc-483c-bd89-e44b8bcbc9f9 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:16.810129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:16:16.815107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ddcfc-703b-44d0-9fb0-c55f4551e8b6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:16.815529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ddcfc-703b-44d0-9fb0-c55f4551e8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:16.815641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ddcfc-703b-44d0-9fb0-c55f4551e8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:16.834616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b44ddcfc-703b-44d0-9fb0-c55f4551e8b6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:16.835800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b44ddcfc-703b-44d0-9fb0-c55f4551e8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:16.844314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:16.994156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3566ff-17cc-483c-bd89-e44b8bcbc9f9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:17.008253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3566ff-17cc-483c-bd89-e44b8bcbc9f9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:17.013642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:17.014326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:17.014605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:17.014876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:17.022896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:17.024459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e792d13-2061-49f0-af5b-ac21e66019cf service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:17.034704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 706/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:17.980730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acbf8-2958-43c6-ae5e-02f06a5fedd2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:17.981296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acbf8-2958-43c6-ae5e-02f06a5fedd2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:17.981296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1acbf8-2958-43c6-ae5e-02f06a5fedd2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:17.982688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:17.987650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06b493-d74f-44a8-a624-b61cafda47bb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:17.988052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06b493-d74f-44a8-a624-b61cafda47bb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:17.988216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06b493-d74f-44a8-a624-b61cafda47bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:18.006744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-be06b493-d74f-44a8-a624-b61cafda47bb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:18.007917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-be06b493-d74f-44a8-a624-b61cafda47bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:18.015551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:18.164288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1acbf8-2958-43c6-ae5e-02f06a5fedd2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:18.178839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1acbf8-2958-43c6-ae5e-02f06a5fedd2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:18.184837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:18.185463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:18.185743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:18.186035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:18.193875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:18.195491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c356-40cb-4f83-a6cb-e55c4fe14b6f service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:18.204610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 724/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:19.106359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee14dc-c82f-4ab2-bf3b-026e8b0e5df8 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:19.106871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee14dc-c82f-4ab2-bf3b-026e8b0e5df8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:19.106871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee14dc-c82f-4ab2-bf3b-026e8b0e5df8 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:19.108228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:19.113105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83a1f9-988d-4e44-96fd-c9bc8bfaed17 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:19.113492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83a1f9-988d-4e44-96fd-c9bc8bfaed17 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:19.113666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83a1f9-988d-4e44-96fd-c9bc8bfaed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:19.131854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-aa83a1f9-988d-4e44-96fd-c9bc8bfaed17 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:19.133040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa83a1f9-988d-4e44-96fd-c9bc8bfaed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:19.141330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:19.288862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee14dc-c82f-4ab2-bf3b-026e8b0e5df8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:19.302874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee14dc-c82f-4ab2-bf3b-026e8b0e5df8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:19.308665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:19.309292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:19.309572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:19.309842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:19.317687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:19.319274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463843db-4370-4752-b1a6-3d2bde69825a service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:19.328932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 709/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:20.261565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707d4d-1fe4-47b0-a5c2-bbc90af99b4a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:20.262054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707d4d-1fe4-47b0-a5c2-bbc90af99b4a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:20.262172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707d4d-1fe4-47b0-a5c2-bbc90af99b4a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:20.263506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:20.268547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50081-634e-4485-9582-eeb7a4924e5e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:20.268950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50081-634e-4485-9582-eeb7a4924e5e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:20.269098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50081-634e-4485-9582-eeb7a4924e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:20.286952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-59f50081-634e-4485-9582-eeb7a4924e5e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:20.288128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59f50081-634e-4485-9582-eeb7a4924e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:20.296814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:20.446136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707d4d-1fe4-47b0-a5c2-bbc90af99b4a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:20.460131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707d4d-1fe4-47b0-a5c2-bbc90af99b4a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:20.465852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:20.466536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:20.466819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:20.467117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:20.475247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:20.476824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29a6bd-f25a-4686-9720-d13119d02cb5 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:20.486526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 727/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:21.398983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c65af-5c87-4592-b49f-7708e2305183 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:21.400005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c65af-5c87-4592-b49f-7708e2305183 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:21.400134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c65af-5c87-4592-b49f-7708e2305183 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:21.401442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:21.406260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7073-8649-4da2-81cd-4253df1a43bb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:21.406651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7073-8649-4da2-81cd-4253df1a43bb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:21.406810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7073-8649-4da2-81cd-4253df1a43bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:21.424800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d66d7073-8649-4da2-81cd-4253df1a43bb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:21.425979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d66d7073-8649-4da2-81cd-4253df1a43bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:21.433635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:21.581666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c65af-5c87-4592-b49f-7708e2305183 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:21.596498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c65af-5c87-4592-b49f-7708e2305183 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:21.601854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:21.602597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:21.602878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:21.603153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:21.611262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:21.612866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d8b9-40a1-4d1c-b1ea-20d980ea6a2f service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:21.622985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 712/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:23.674000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9e646-b1fb-4d78-b45c-0f980ab0853d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:23.674567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9e646-b1fb-4d78-b45c-0f980ab0853d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:23.674685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9e646-b1fb-4d78-b45c-0f980ab0853d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:23.675987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:23.681456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882ccf-4fa5-4658-a6ff-60309084185e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:23.681872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882ccf-4fa5-4658-a6ff-60309084185e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:23.682864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882ccf-4fa5-4658-a6ff-60309084185e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:23.700598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1e882ccf-4fa5-4658-a6ff-60309084185e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:23.701772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e882ccf-4fa5-4658-a6ff-60309084185e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:23.709753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:23.747718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9e646-b1fb-4d78-b45c-0f980ab0853d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:23.758263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9e646-b1fb-4d78-b45c-0f980ab0853d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:23.762401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:23.763038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:23.763324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:23.763603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:23.772285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:23.773719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a00f34-1e72-4a7a-9a32-77a05de1294a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:23.783007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 730/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:26.101991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f326-ac0b-41a0-b09a-7e72c1551ddc None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:26.102565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f326-ac0b-41a0-b09a-7e72c1551ddc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:26.102687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f326-ac0b-41a0-b09a-7e72c1551ddc None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:26.104001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:26.109487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c1b7-ba1d-40cd-b635-502e1990338f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:26.109916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c1b7-ba1d-40cd-b635-502e1990338f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:26.110047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c1b7-ba1d-40cd-b635-502e1990338f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:26.127318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1f51c1b7-ba1d-40cd-b635-502e1990338f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:26.128511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1f51c1b7-ba1d-40cd-b635-502e1990338f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:26.136664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:26.173427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f326-ac0b-41a0-b09a-7e72c1551ddc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:26.183852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f326-ac0b-41a0-b09a-7e72c1551ddc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:26.187333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:26.187958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:26.188241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:26.188514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:26.197136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:26.199111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a31e1-52b1-4692-85ef-6dda5808c897 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:26.208131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 715/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:28.764285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50495-272a-4171-96f0-902886ab0bbc None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:28.764873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50495-272a-4171-96f0-902886ab0bbc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:28.764873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a50495-272a-4171-96f0-902886ab0bbc None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:28.766232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:28.771537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c8a0-4d0d-4e1a-ab30-1f8062dbde44 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:28.771981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c8a0-4d0d-4e1a-ab30-1f8062dbde44 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:28.772089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c8a0-4d0d-4e1a-ab30-1f8062dbde44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:28.789796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2e07c8a0-4d0d-4e1a-ab30-1f8062dbde44 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:28.791110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2e07c8a0-4d0d-4e1a-ab30-1f8062dbde44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:28.799484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:28.837778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50495-272a-4171-96f0-902886ab0bbc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:28.849081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a50495-272a-4171-96f0-902886ab0bbc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:28.852573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:28.853197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:28.853479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:28.854423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:28.863127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:28.864554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7586-c8b5-42b8-b110-9a5754d5b256 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:28.874146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 733/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:31.182094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d57dc-607b-4270-9e56-442e43cc7fac None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:31.182648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d57dc-607b-4270-9e56-442e43cc7fac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:31.182648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d57dc-607b-4270-9e56-442e43cc7fac None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:31.184005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:31.189192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbdd0e-549d-4f4c-88ac-2039a36217c8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:31.189647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbdd0e-549d-4f4c-88ac-2039a36217c8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:31.189763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbdd0e-549d-4f4c-88ac-2039a36217c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:31.207033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-45bbdd0e-549d-4f4c-88ac-2039a36217c8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:31.208230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-45bbdd0e-549d-4f4c-88ac-2039a36217c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:31.216771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:31.254959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d57dc-607b-4270-9e56-442e43cc7fac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:31.265421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d57dc-607b-4270-9e56-442e43cc7fac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:31.268899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:31.269528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:31.269810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:31.270120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:31.279241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:31.280650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447246e-89a5-4f78-8352-c7ea562468de service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:31.289771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 718/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:33.519649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb04ac7-d9f1-45b0-ad43-63caffc65288 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:33.520177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb04ac7-d9f1-45b0-ad43-63caffc65288 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:33.520177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb04ac7-d9f1-45b0-ad43-63caffc65288 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:33.521547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:33.526923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4dacc-ce6c-41d5-a6aa-46e06ee2c1d1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:33.527362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4dacc-ce6c-41d5-a6aa-46e06ee2c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:33.527519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4dacc-ce6c-41d5-a6aa-46e06ee2c1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:33.544959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f8d4dacc-ce6c-41d5-a6aa-46e06ee2c1d1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:33.546201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f8d4dacc-ce6c-41d5-a6aa-46e06ee2c1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:33.553900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:33.592577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb04ac7-d9f1-45b0-ad43-63caffc65288 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:33.603070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb04ac7-d9f1-45b0-ad43-63caffc65288 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:33.607259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:33.607899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:33.608183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:33.608457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:33.617445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:33.618906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f16aa-4750-4167-97fe-14538d8cef4f service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:33.628986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 736/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:35.848263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f114f-e1e0-4fcd-a7d8-395ae054b180 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:35.848849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f114f-e1e0-4fcd-a7d8-395ae054b180 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:35.848849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f114f-e1e0-4fcd-a7d8-395ae054b180 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:35.850209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:35.856027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e990-4c1f-4c59-96c7-4f37016c018f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:35.856417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e990-4c1f-4c59-96c7-4f37016c018f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:35.856578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e990-4c1f-4c59-96c7-4f37016c018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:35.874774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fba6e990-4c1f-4c59-96c7-4f37016c018f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:35.875953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fba6e990-4c1f-4c59-96c7-4f37016c018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:35.883584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:35.920760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f114f-e1e0-4fcd-a7d8-395ae054b180 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:35.932183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f114f-e1e0-4fcd-a7d8-395ae054b180 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:35.935728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:35.936367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:35.936647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:35.937028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:35.945795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:35.947322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55314a46-e061-4128-903c-c903a7dd1d2f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:35.956472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 721/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:38.201599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8f97-2861-497e-8bd7-029e71a840cf None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:38.202189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8f97-2861-497e-8bd7-029e71a840cf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:38.202189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8f97-2861-497e-8bd7-029e71a840cf None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:38.203562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:38.208724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc2b1-d417-4419-830b-7c9a7ba31960 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:38.209162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc2b1-d417-4419-830b-7c9a7ba31960 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:38.209260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc2b1-d417-4419-830b-7c9a7ba31960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:38.215695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=70 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:16:38.227522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2cfdc2b1-d417-4419-830b-7c9a7ba31960 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:38.229408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2cfdc2b1-d417-4419-830b-7c9a7ba31960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:38.237136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:38.274264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc8f97-2861-497e-8bd7-029e71a840cf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:38.279633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=73 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:16:38.285350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc8f97-2861-497e-8bd7-029e71a840cf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:38.288884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:38.289514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:38.289792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:38.290098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:38.299442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:38.300901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9e850-f4a0-45ac-8bb6-3919cd82610a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:38.311073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 739/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:40.538687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc5a85-d9bd-4fe0-812a-91d775bfd69a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:40.539221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc5a85-d9bd-4fe0-812a-91d775bfd69a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:40.539221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc5a85-d9bd-4fe0-812a-91d775bfd69a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:40.540547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:40.545803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b135e5-b778-468f-a166-2096c9d3b555 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:40.546305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b135e5-b778-468f-a166-2096c9d3b555 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:40.546397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b135e5-b778-468f-a166-2096c9d3b555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:40.563652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b8b135e5-b778-468f-a166-2096c9d3b555 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:40.564830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b8b135e5-b778-468f-a166-2096c9d3b555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:40.572925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:40.610602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc5a85-d9bd-4fe0-812a-91d775bfd69a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:40.620999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc5a85-d9bd-4fe0-812a-91d775bfd69a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:40.624425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:40.625047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:40.625326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:40.625596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:40.634204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:40.636487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f719a56-3ae3-4c6d-a72a-f6f2236e8a6f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:40.645609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 724/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:42.834476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2770b4-4822-45f3-9e0d-769aea5910d1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:42.835039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2770b4-4822-45f3-9e0d-769aea5910d1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:42.835039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2770b4-4822-45f3-9e0d-769aea5910d1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:42.836382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:42.842537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aee6-9c25-439a-98a9-3fb8a9d69109 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:42.843002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aee6-9c25-439a-98a9-3fb8a9d69109 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:42.843113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791aee6-9c25-439a-98a9-3fb8a9d69109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:42.860403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7791aee6-9c25-439a-98a9-3fb8a9d69109 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:42.861578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7791aee6-9c25-439a-98a9-3fb8a9d69109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:42.869218 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:42.905070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2770b4-4822-45f3-9e0d-769aea5910d1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:42.915505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2770b4-4822-45f3-9e0d-769aea5910d1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:42.919010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:42.919622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:42.919905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:42.920173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:42.928719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:42.930168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9bd2e-6f77-4d44-93a5-64d301c6c452 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:42.939750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 742/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:45.139236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e185-7ec5-4cc1-a831-a74a9fcc5691 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:45.139798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e185-7ec5-4cc1-a831-a74a9fcc5691 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:45.139798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e185-7ec5-4cc1-a831-a74a9fcc5691 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:45.141150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:16:45.146350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc319-d9e6-4e91-8058-b8d558eaf570 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:45.146761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc319-d9e6-4e91-8058-b8d558eaf570 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:45.146900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc319-d9e6-4e91-8058-b8d558eaf570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:45.164788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6e8dc319-d9e6-4e91-8058-b8d558eaf570 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:16:45.166010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6e8dc319-d9e6-4e91-8058-b8d558eaf570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:16:45.173702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:16:45.209969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696e185-7ec5-4cc1-a831-a74a9fcc5691 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:16:45.221568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696e185-7ec5-4cc1-a831-a74a9fcc5691 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:16:45.225401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:16:45.226083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:16:45.226396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:16:45.226673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:16:45.235873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:16:45.237287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be71decf-f9ee-4f76-a98c-5a9d384791ca service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:16:45.247094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 727/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:16:55.159759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=17 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:16:55.242427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=17 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:17:18.294522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c978581-10ff-4f74-81b9-724e082109b0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:18.295083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c978581-10ff-4f74-81b9-724e082109b0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:18.295083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c978581-10ff-4f74-81b9-724e082109b0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:18.296446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 744/1471] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:18.301662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854523-f7e0-4668-9c5b-2dc2e0966a51 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:18.302111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854523-f7e0-4668-9c5b-2dc2e0966a51 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:18.302352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854523-f7e0-4668-9c5b-2dc2e0966a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:18.320099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f4854523-f7e0-4668-9c5b-2dc2e0966a51 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:18.321879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f4854523-f7e0-4668-9c5b-2dc2e0966a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:18.329438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 728/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:18.375668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e89e-4386-49ae-b326-448f16a87b7f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:18.375953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e89e-4386-49ae-b326-448f16a87b7f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:18.376228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e89e-4386-49ae-b326-448f16a87b7f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:18.377563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 745/1473] 10.222.0.16 () {54 vars in 962 bytes} [Mon May 23 20:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:18.382137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7faec9-e65c-4921-84ab-077cc0ac98ef None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:18.382428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7faec9-e65c-4921-84ab-077cc0ac98ef None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:18.382731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7faec9-e65c-4921-84ab-077cc0ac98ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:18.400410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3c7faec9-e65c-4921-84ab-077cc0ac98ef None None] MFA Rules not processed for user `bef73ca7ffe94b79b085dcfa1eae278d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:18.401570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3c7faec9-e65c-4921-84ab-077cc0ac98ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:18.407834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 729/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Mon May 23 20:17:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:18.414678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6e89e-4386-49ae-b326-448f16a87b7f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:18.424954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6e89e-4386-49ae-b326-448f16a87b7f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:18.428490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:18.429100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:18.429377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:18.429647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:18.438807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:18.440223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d83cec-9368-48f6-b7bf-e4702058a5af service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:18.449907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 746/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:17:20.710323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67169a-75cf-48c5-8aa9-8011664806c6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:20.710850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67169a-75cf-48c5-8aa9-8011664806c6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:20.710850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67169a-75cf-48c5-8aa9-8011664806c6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:20.712199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 730/1476] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:20.717625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef97aa-f8a1-40c1-be9c-3b49da8dc5c1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:20.718097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef97aa-f8a1-40c1-be9c-3b49da8dc5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:20.718203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef97aa-f8a1-40c1-be9c-3b49da8dc5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:20.735315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-19ef97aa-f8a1-40c1-be9c-3b49da8dc5c1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:20.736497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-19ef97aa-f8a1-40c1-be9c-3b49da8dc5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:20.744598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 747/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:20.767908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67169a-75cf-48c5-8aa9-8011664806c6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:20.780619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67169a-75cf-48c5-8aa9-8011664806c6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:20.785240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:20.785882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:20.786205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:20.786484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:20.787219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:20.788977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:17:20.788977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:17:20.788977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:17:20.790136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4c616-68b1-400d-aa6f-7b65bdc6f0ff admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:20.809340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 731/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 20:17:20 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:17:23.002937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8f68-8710-4da6-8fdb-38b3d8e383b4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:23.003512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8f68-8710-4da6-8fdb-38b3d8e383b4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:23.003512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8f68-8710-4da6-8fdb-38b3d8e383b4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:23.004907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 748/1479] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:23.010229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed006-9d8a-4b2e-af70-de704ece49c2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:23.010676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed006-9d8a-4b2e-af70-de704ece49c2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:23.010769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed006-9d8a-4b2e-af70-de704ece49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:23.028778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a08ed006-9d8a-4b2e-af70-de704ece49c2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:23.029957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a08ed006-9d8a-4b2e-af70-de704ece49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:23.037662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 732/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:23.080442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d8f68-8710-4da6-8fdb-38b3d8e383b4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:23.095413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d8f68-8710-4da6-8fdb-38b3d8e383b4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:23.100777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:23.101405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:23.101726 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:23.102007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:23.109959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:23.112513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ce7fe-9d78-4557-a031-21cf4d986d62 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:23.120860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 749/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:17:25.505917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8be89-b6e1-451e-adbf-97f24d619a64 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:25.506509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8be89-b6e1-451e-adbf-97f24d619a64 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:25.506509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8be89-b6e1-451e-adbf-97f24d619a64 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:25.507909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 733/1482] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:25.512987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328a1c-994f-4316-b155-64a3db2a42eb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:25.513416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328a1c-994f-4316-b155-64a3db2a42eb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:25.513544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328a1c-994f-4316-b155-64a3db2a42eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:25.530688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cd328a1c-994f-4316-b155-64a3db2a42eb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:25.531874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cd328a1c-994f-4316-b155-64a3db2a42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:25.540024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 750/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:25.575830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8be89-b6e1-451e-adbf-97f24d619a64 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:25.586040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8be89-b6e1-451e-adbf-97f24d619a64 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:25.589838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:25.590505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:25.590795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:25.591072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:25.600284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:25.602245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22d31-82ae-4b4c-b9a5-215b6e2e157c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:25.611464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 734/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:17:27.837288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa85086-909b-40b2-9d04-446b29a8ce4a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:27.838133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa85086-909b-40b2-9d04-446b29a8ce4a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:27.838133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa85086-909b-40b2-9d04-446b29a8ce4a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:27.839496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 751/1485] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:27.844591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341795c-d480-459a-aad0-dd2f69ddb44b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:27.845003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341795c-d480-459a-aad0-dd2f69ddb44b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:27.845219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341795c-d480-459a-aad0-dd2f69ddb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:27.864171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a341795c-d480-459a-aad0-dd2f69ddb44b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:27.865368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a341795c-d480-459a-aad0-dd2f69ddb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:27.873115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 735/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:17:27.911322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa85086-909b-40b2-9d04-446b29a8ce4a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:27.921697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa85086-909b-40b2-9d04-446b29a8ce4a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:27.925118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:27.925741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:27.926038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:27.926323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:27.934848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:27.936269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b13d0-f333-4443-b4f9-db09807f34a3 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:27.945871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 752/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:17:30.272198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a872da-74d6-4802-86cd-d43622c8f152 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:30.272693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a872da-74d6-4802-86cd-d43622c8f152 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:30.272796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a872da-74d6-4802-86cd-d43622c8f152 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:30.274666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 736/1488] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:30.280004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab75fd-51c1-461a-8329-d0bd208ba853 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:30.280427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab75fd-51c1-461a-8329-d0bd208ba853 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:30.280552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab75fd-51c1-461a-8329-d0bd208ba853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:30.298660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fdab75fd-51c1-461a-8329-d0bd208ba853 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:30.299854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fdab75fd-51c1-461a-8329-d0bd208ba853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:30.307561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 753/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:30.347417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a872da-74d6-4802-86cd-d43622c8f152 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:30.358548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a872da-74d6-4802-86cd-d43622c8f152 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:30.362003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:30.362663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:30.362952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:30.363228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:30.371887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:30.373292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3833e-8bbf-4dd4-a143-a096fe8487a3 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:30.382405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 737/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:17:33.168424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549287-0b12-4969-ac50-d1daf39ee7d0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:33.169031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549287-0b12-4969-ac50-d1daf39ee7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:33.169031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549287-0b12-4969-ac50-d1daf39ee7d0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:33.170362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 754/1491] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 20:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:17:33.175626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769527ca-4aac-48d5-b046-50c93df05fc6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:33.176062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769527ca-4aac-48d5-b046-50c93df05fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:33.176166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769527ca-4aac-48d5-b046-50c93df05fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:33.193443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-769527ca-4aac-48d5-b046-50c93df05fc6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:17:33.194659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-769527ca-4aac-48d5-b046-50c93df05fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:17:33.202903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 738/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 20:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:17:33.241364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31549287-0b12-4969-ac50-d1daf39ee7d0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:17:33.252432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31549287-0b12-4969-ac50-d1daf39ee7d0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:17:33.255914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:17:33.256537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:17:33.256814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:17:33.257083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:17:33.266421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:17:33.267839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672b90c-e7ed-4ffe-95df-356715e5dfbd service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:17:33.277418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 755/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:17:43.188498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=33 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:17:43.273331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=33 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:19:21.450913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.451499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.451499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.469627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:19:21.470849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:21.479188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 739/1494] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:19:21.488017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.488520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.488622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.506121 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:19:21.507317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:21.515046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 756/1495] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:19:21.519817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.529044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23c7ff-e63a-4b0b-886b-84ded5a4bc15 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.533486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.534169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.534449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.534720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.535982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:19:21.536281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.537272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.537272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:19:21.537272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:19:21.538169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:21.549515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 740/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:19:21.557510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.567023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668203e-5c53-4eb6-a010-3f6d12fedaa7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.571589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.572250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.572534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.572809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.573500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.575306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.575306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:19:21.575306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:19:21.576509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:21.708866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 757/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:19:21 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:19:21.713809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.723998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544fc4d-548e-4deb-9e8e-969f2ad563d7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.777941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.778564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.778846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.779120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.779796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.780783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.780783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:19:21.780783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:19:21.781973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:21.866957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 741/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:19:21 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:19:21.871193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.881956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aeacf-10ba-41dc-9528-fb2e68b60860 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.886679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.887361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.887656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.887914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.888683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.889690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.889690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:19:21.889690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:19:21.890684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:21.903828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 758/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:19:21.909559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.919316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4331d-5662-4e05-8abc-37d41ffeabc2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.923779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.924457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.924734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.925003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.925718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.926797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.926797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:19:21.926797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:19:21.927662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d24516-dffc-4a80-b5cb-5fb2e4823dbc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:21.940722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 742/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:19:21.945222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:21.955127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1c40-7b4c-413a-943c-d622bfae578e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:21.960765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:21.961653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:21.961968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:21.962278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] PATH_INFO: `/v3/projects/2eb2087225c141ebab28386b60f175b0/users/1b9df74d28a74570be64b92d95face1a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:21.977158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb2087225c141ebab28386b60f175b0, user_id=1b9df74d28a74570be64b92d95face1a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:21.978238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:21.978238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:19:21.978238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:19:21.981658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac1267-1f3f-42ca-9468-34538c55a793 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:22.032481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 759/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:19:21 2022] PUT /identity/v3/projects/2eb2087225c141ebab28386b60f175b0/users/1b9df74d28a74570be64b92d95face1a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:19:22.043761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bebd06-49e7-4fe2-9283-989a8c1b1d9a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.044051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bebd06-49e7-4fe2-9283-989a8c1b1d9a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.044327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bebd06-49e7-4fe2-9283-989a8c1b1d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.070481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d6bebd06-49e7-4fe2-9283-989a8c1b1d9a None None] MFA Rules not processed for user `1b9df74d28a74570be64b92d95face1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:19:22.071663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d6bebd06-49e7-4fe2-9283-989a8c1b1d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:22.133798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 743/1502] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:19:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:19:22.146979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3131-0bf1-412c-9418-7a2dbd4ddf3b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.147274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3131-0bf1-412c-9418-7a2dbd4ddf3b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.147560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce3131-0bf1-412c-9418-7a2dbd4ddf3b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.148959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 760/1503] 10.222.0.16 () {54 vars in 963 bytes} [Mon May 23 20:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:19:22.153733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9258-ef90-4003-a31c-62aeefe99a8e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.154039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9258-ef90-4003-a31c-62aeefe99a8e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.154328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9258-ef90-4003-a31c-62aeefe99a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.171372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f39d9258-ef90-4003-a31c-62aeefe99a8e None None] MFA Rules not processed for user `567602cb67c447c798d4009f77140b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:19:22.172548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f39d9258-ef90-4003-a31c-62aeefe99a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:22.215188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:19:22.222542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 744/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Mon May 23 20:19:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:19:22.230416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3131-0bf1-412c-9418-7a2dbd4ddf3b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:22.241352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce3131-0bf1-412c-9418-7a2dbd4ddf3b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:22.244957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:22.245585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.245864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.246218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.255264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:22.256681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:22.265332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 761/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:19:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:19:22.307782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8718e-104e-4994-8186-8d277f7be056 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.308071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8718e-104e-4994-8186-8d277f7be056 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.308349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8718e-104e-4994-8186-8d277f7be056 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.309671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 745/1506] 10.222.0.16 () {52 vars in 829 bytes} [Mon May 23 20:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:19:22.313103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:22.376841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5988-5fcc-4677-9672-0c466cb2b807 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:22.381642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:22.382646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.382997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.383342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] PATH_INFO: `/v3/users/1b9df74d28a74570be64b92d95face1a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.384600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9df74d28a74570be64b92d95face1a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:22.385690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:22.385690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:19:22.385690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:19:22.387028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:22.599657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9df74d28a74570be64b92d95face1a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:22.600294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9df74d28a74570be64b92d95face1a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:22.600874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9df74d28a74570be64b92d95face1a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:22.619208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9df74d28a74570be64b92d95face1a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:22.631014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9df74d28a74570be64b92d95face1a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:22.634951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:19:22.727879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 762/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:19:22 2022] DELETE /identity/v3/users/1b9df74d28a74570be64b92d95face1a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:19:22.732315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.732814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.732911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.749878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:19:22.751111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:22.807511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 746/1508] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:19:22.816803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:22.818260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:22.873350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769d42-eeb1-46f7-b094-7044431a4bf1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:22.876786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:22.877421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:22.877700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:22.877971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:22.886662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:22.890056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff439ba-9364-48c8-a3fa-cc409cb9910b service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:22.897703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 763/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:19:23.406116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:19:23.407683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:19:23.419546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6306d-e6ba-4d4a-80ce-2ca93a63676d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:19:23.424140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:19:23.425006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:19:23.425293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:19:23.425567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] PATH_INFO: `/v3/projects/2eb2087225c141ebab28386b60f175b0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:19:23.426764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb2087225c141ebab28386b60f175b0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:19:23.427777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:19:23.427777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:19:23.427777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:19:23.428970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:19:23.558195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb2087225c141ebab28386b60f175b0', 'request_id': 'req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:23.632410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb2087225c141ebab28386b60f175b0', 'request_id': 'req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:23.638679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb2087225c141ebab28386b60f175b0', 'request_id': 'req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:23.657489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] The token cache is being invalidate because project 2eb2087225c141ebab28386b60f175b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:19:23.657686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:23.658406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-dd829b2e-1b96-4d3e-9ec9-b1af7881b1c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:19:23.660073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 747/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:19:23 2022] DELETE /identity/v3/projects/2eb2087225c141ebab28386b60f175b0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:19:32.893133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:19:33.654114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:20:58.634647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:58.636218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:58.636218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:58.636577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:20:58.641256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979ad0-bcb4-4106-bfe4-0c66d48bd8ca None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:58.642230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979ad0-bcb4-4106-bfe4-0c66d48bd8ca None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:58.642532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea979ad0-bcb4-4106-bfe4-0c66d48bd8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:58.660375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ea979ad0-bcb4-4106-bfe4-0c66d48bd8ca None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:20:58.661566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ea979ad0-bcb4-4106-bfe4-0c66d48bd8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:20:58.718567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:20:58.893701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:20:58.893701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:20:58.966694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6967a-31a3-40d3-af2a-cf07b8296154 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:20:58.972402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:20:58.973099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:58.973382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:58.973658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:58.981919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:20:58.983544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ece3-6404-445c-a2be-72b9dad58529 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:20:58.993972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 765/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:20:59.814552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca7aa5-f0c8-4215-83e1-899a0cbe9439 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:59.815107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca7aa5-f0c8-4215-83e1-899a0cbe9439 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:59.815107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca7aa5-f0c8-4215-83e1-899a0cbe9439 None None] PATH_INFO: `/v3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:59.816437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 749/1514] 10.222.0.31 () {54 vars in 925 bytes} [Mon May 23 20:20:59 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:20:59.824036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:59.824538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:59.824640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:59.843531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:20:59.844770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:20:59.878131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3ca6-f410-45b9-acaa-9e69e3e3aa50 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:59.878425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3ca6-f410-45b9-acaa-9e69e3e3aa50 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:59.878705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da3ca6-f410-45b9-acaa-9e69e3e3aa50 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:59.882090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 750/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:20:59.884888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b1399-de90-44af-8413-55253444acef None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:20:59.885244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b1399-de90-44af-8413-55253444acef None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:20:59.885596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b1399-de90-44af-8413-55253444acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:20:59.903695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-be8b1399-de90-44af-8413-55253444acef None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:20:59.904889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-be8b1399-de90-44af-8413-55253444acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:20:59.908374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 766/1516] 10.222.0.31 () {58 vars in 1015 bytes} [Mon May 23 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:20:59.913127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 751/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:00.063904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:00.075774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a243037-aaa2-4a71-8d29-635ac2d99191 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:00.081639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:00.082312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:00.082591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:00.082864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:00.090987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:00.092559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691c67e-cf44-41a7-bc09-f0f9583f2cfd service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:00.101683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 767/1518] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:03.954210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddecc0-b11e-4492-a04e-8d43e28c19ec None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:03.954732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddecc0-b11e-4492-a04e-8d43e28c19ec None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:03.954840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddecc0-b11e-4492-a04e-8d43e28c19ec None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:03.956120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 752/1519] 10.222.0.16 () {54 vars in 960 bytes} [Mon May 23 20:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:03.961363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44457-79f2-4aaa-83c8-75c323743c06 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:03.961760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44457-79f2-4aaa-83c8-75c323743c06 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:03.961909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44457-79f2-4aaa-83c8-75c323743c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:03.983317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-49d44457-79f2-4aaa-83c8-75c323743c06 None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:03.984515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-49d44457-79f2-4aaa-83c8-75c323743c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:03.992816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 768/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Mon May 23 20:21:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:04.000476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddecc0-b11e-4492-a04e-8d43e28c19ec None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:04.013848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddecc0-b11e-4492-a04e-8d43e28c19ec None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:04.019436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:04.020079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:04.020355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:04.020654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:04.028856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:04.030488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01495671-e05b-43a4-93a0-21e51284452c service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:04.040555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 753/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:11.378079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c383dad-88ef-44b7-ad96-47fe3447e8ef None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:11.378653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c383dad-88ef-44b7-ad96-47fe3447e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:11.378653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c383dad-88ef-44b7-ad96-47fe3447e8ef None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:11.380058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:11.384862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3313f8f-bb5f-48ef-bbe6-5d7aa0a679d9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:11.385331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3313f8f-bb5f-48ef-bbe6-5d7aa0a679d9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:11.385432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3313f8f-bb5f-48ef-bbe6-5d7aa0a679d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:11.403035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c3313f8f-bb5f-48ef-bbe6-5d7aa0a679d9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:11.404220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c3313f8f-bb5f-48ef-bbe6-5d7aa0a679d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:11.412481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:11.564008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c383dad-88ef-44b7-ad96-47fe3447e8ef None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:11.578468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c383dad-88ef-44b7-ad96-47fe3447e8ef None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:11.584069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:11.584693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:11.585000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:11.585274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:11.593160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:11.595446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ae1b-a5e4-4447-9596-a9d37f390f49 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:11.604638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 770/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:12.459975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682cfe-1ee2-4f50-a1f1-34e436fa45d0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:12.460557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682cfe-1ee2-4f50-a1f1-34e436fa45d0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:12.460557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682cfe-1ee2-4f50-a1f1-34e436fa45d0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:12.461916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 20:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:12.466729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124230-5610-4948-9dee-738c2cccf10c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:12.467160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124230-5610-4948-9dee-738c2cccf10c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:12.467287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124230-5610-4948-9dee-738c2cccf10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:12.485293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-da124230-5610-4948-9dee-738c2cccf10c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:12.486469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da124230-5610-4948-9dee-738c2cccf10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:12.494336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 20:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:12.643493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682cfe-1ee2-4f50-a1f1-34e436fa45d0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:12.657947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682cfe-1ee2-4f50-a1f1-34e436fa45d0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:12.663272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:12.663903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:12.664187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:12.664463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:12.672999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:12.674597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98994c-85cf-436b-a76b-3c68c5859daf service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:12.683736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 756/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:16.725086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:16.725752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:16.725752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:16.727020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:16.732088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218fec-3b23-4170-a56e-f579d14dccb3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:16.732486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218fec-3b23-4170-a56e-f579d14dccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:16.732641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218fec-3b23-4170-a56e-f579d14dccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:16.750375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5b218fec-3b23-4170-a56e-f579d14dccb3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:16.751567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5b218fec-3b23-4170-a56e-f579d14dccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:16.759267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:21:16.795657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:16.797341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:16.842830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:21:16.854772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd86f99-b7db-4257-bdf6-cb66729aaaf6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:16.858292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:16.858929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:16.859214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:16.859486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:16.868146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:16.869567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce16b7c-0113-4bca-be94-1f0ace30cab7 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:16.879142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 773/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:19.071859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b74-8542-4cf5-84f4-82a370965834 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:19.072433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b74-8542-4cf5-84f4-82a370965834 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:19.072433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23b74-8542-4cf5-84f4-82a370965834 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:19.073731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:19.079757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c959b-85d6-47c2-8728-a52932fcafb7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:19.080210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c959b-85d6-47c2-8728-a52932fcafb7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:19.080309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c959b-85d6-47c2-8728-a52932fcafb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:19.099167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6a9c959b-85d6-47c2-8728-a52932fcafb7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:19.100449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6a9c959b-85d6-47c2-8728-a52932fcafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:19.108118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:19.144427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23b74-8542-4cf5-84f4-82a370965834 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:19.155184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23b74-8542-4cf5-84f4-82a370965834 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:19.159228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:19.159897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:19.160181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:19.160461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:19.169292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:19.170755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e72188-a1e3-4bbe-b817-d9d556ec8050 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:19.180016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 759/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:22.859489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:22.860104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:22.860104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:22.861424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:21:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:22.866755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29dc4-61cf-4bec-807f-889a45a3feaa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:22.867234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29dc4-61cf-4bec-807f-889a45a3feaa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:22.867328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29dc4-61cf-4bec-807f-889a45a3feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:22.886668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-77b29dc4-61cf-4bec-807f-889a45a3feaa None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:22.887868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-77b29dc4-61cf-4bec-807f-889a45a3feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:22.902130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:23.167346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:23.169010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:23.227314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0546255-cd0d-458f-9b48-2a8384cc8df3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:23.230884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:23.231531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:23.231812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:23.232088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:23.241499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:23.242958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1481e-cd6c-447f-b1eb-b1fe28e8a024 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:23.252197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 776/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:25.465304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:25.465869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:25.465869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:25.467293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 20:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:21:25.472631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496d6d-0262-43a4-98e6-89b21e14fe34 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:25.473026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496d6d-0262-43a4-98e6-89b21e14fe34 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:25.473194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496d6d-0262-43a4-98e6-89b21e14fe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:25.490853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-86496d6d-0262-43a4-98e6-89b21e14fe34 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:21:25.492044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-86496d6d-0262-43a4-98e6-89b21e14fe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:25.500348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 20:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:21:25.766958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:21:25.768678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:21:25.776175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=58 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:21:25.792261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dacb6-57c5-4c03-8798-71f9558487e8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:21:25.792261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:21:25.792261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:21:25.792261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:21:25.792630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:21:25.796604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:21:25.798050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceae7d7-5fdc-480f-bf2f-694656471149 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:21:25.807282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 762/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:21:35.485726 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=39 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:21:35.810378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:53.604294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.605510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.606030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.636289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:53.638515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:53.654767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 778/1540] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:53.664758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.665837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.666466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.695290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:53.696899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:53.706448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 763/1541] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:53.711650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:53.723382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea0607-d1fd-4b5e-8a06-ef843d327d60 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:53.732640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:53.733719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.734257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.734755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.735905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:53.736440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:53.737713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 20:22:53.737713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:22:53.737713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:22:53.739065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73209b-d501-48e5-9567-9299a858f736 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:53.756970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 779/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:53.766071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:53.778123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45ab76-4daa-4dbb-8de4-865f81dc5df6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:53.786494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:53.787574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.788091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.788586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.789520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:53.790857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:22:53.790857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:22:53.790857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:22:53.792512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:53.818107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b61a0-c3e9-4b32-8b5c-87711bc9e53c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.819047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b61a0-c3e9-4b32-8b5c-87711bc9e53c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.819555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b61a0-c3e9-4b32-8b5c-87711bc9e53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.855262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d93b61a0-c3e9-4b32-8b5c-87711bc9e53c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:53.856846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d93b61a0-c3e9-4b32-8b5c-87711bc9e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:53.868496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 780/1543] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:22:53.880712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1782b-4357-4405-ae89-a807ecb893cf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:53.881451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1782b-4357-4405-ae89-a807ecb893cf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:53.881937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1782b-4357-4405-ae89-a807ecb893cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:53.889424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 764/1544] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:53.904074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dfa1782b-4357-4405-ae89-a807ecb893cf None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:53.905563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dfa1782b-4357-4405-ae89-a807ecb893cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:53.910348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.004807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 781/1545] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:22:54.006581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.007149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.007637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.033420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.035031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.046262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0339-01d3-4e47-88f0-23b8c8cd4f7c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.055989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.056698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.056985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.057265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.057989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.059439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 782/1546] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.060768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9266df-085b-4f50-b0d9-f3451a1a6c54 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.061267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.076576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f78e38-0c41-4429-b4d9-e96f4f2cbf52 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.089514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.093455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.093455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.093455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.093455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:54.093455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.095401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1417b6-5856-470c-9c43-a22a739097a7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.116000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 783/1547] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:54.117669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.118329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.118825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.144892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.146483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.161753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 784/1548] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.163161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.171989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 765/1549] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:53 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:54.173346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52436-ae95-4ef9-af07-5e1b15ccb7a8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.173638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52436-ae95-4ef9-af07-5e1b15ccb7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.173920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52436-ae95-4ef9-af07-5e1b15ccb7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.174550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68ddb1-ad60-4730-8e89-7e86e614a97c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.191393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.192129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.192541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.192863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.193614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.198358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d89097-16c9-47b2-8717-1ffff95ce5fb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.219488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f0d52436-ae95-4ef9-af07-5e1b15ccb7a8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.220746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f0d52436-ae95-4ef9-af07-5e1b15ccb7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.238928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 766/1550] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.240438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.240767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.241040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.263273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.265282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.272589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 767/1551] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.276917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.282687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ee8c7-2a28-4292-a28b-c4712ae9d203 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.287158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.287801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.288082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.288359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.289080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:54.289385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.291008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.302979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 768/1552] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:54.303766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.313545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc870e-5a4c-47eb-a172-9ff43dae399f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.318027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.318656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.318940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.319214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.319917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.321570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787f8e5-3afa-4391-b39b-e2db5c3e1024 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.331455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 785/1553] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:54.334684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 769/1554] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:54.334684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822ce9-e132-4c9e-b54b-d321b78910d0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.340153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011d0c-5df9-426d-bfda-c7143294ed19 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.340253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822ce9-e132-4c9e-b54b-d321b78910d0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.340335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011d0c-5df9-426d-bfda-c7143294ed19 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.340416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59822ce9-e132-4c9e-b54b-d321b78910d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.340499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011d0c-5df9-426d-bfda-c7143294ed19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.352132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-59822ce9-e132-4c9e-b54b-d321b78910d0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.352510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e1011d0c-5df9-426d-bfda-c7143294ed19 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.353335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59822ce9-e132-4c9e-b54b-d321b78910d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.353675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e1011d0c-5df9-426d-bfda-c7143294ed19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.410912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 770/1555] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.412096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 786/1556] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.412213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549be-174e-4c73-89b0-2615946b6255 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.413378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a598a-b950-43f9-8f9f-3fd18d063e6f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.413921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a598a-b950-43f9-8f9f-3fd18d063e6f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.414174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549be-174e-4c73-89b0-2615946b6255 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.414302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a598a-b950-43f9-8f9f-3fd18d063e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.414756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538549be-174e-4c73-89b0-2615946b6255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.432782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-756a598a-b950-43f9-8f9f-3fd18d063e6f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.434148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-756a598a-b950-43f9-8f9f-3fd18d063e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.437451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-538549be-174e-4c73-89b0-2615946b6255 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.438706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-538549be-174e-4c73-89b0-2615946b6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.442942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 787/1557] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.444174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.444449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.444727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.452144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 771/1558] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.453767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538549be-174e-4c73-89b0-2615946b6255 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.465474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.465974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538549be-174e-4c73-89b0-2615946b6255 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.466801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.470759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.471421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.471921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.472042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.472634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.474580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1b9fe-ed3d-427c-b0b0-9dca2acb7e8e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.474690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 788/1559] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.478754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.485603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9be84-60ad-44ff-938f-08e63a13f1a9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.490217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:54.490370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.490906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.491185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.491457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.492140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.493146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:22:54.493146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:22:54.493146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:22:54.494652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.504854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:54.572445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 789/1560] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:54.573178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.577378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 772/1561] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:54.577378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf53a-4661-4833-836d-0b4cb6543beb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.577378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf53a-4661-4833-836d-0b4cb6543beb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.577378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf53a-4661-4833-836d-0b4cb6543beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.583527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2b921-46e3-4a9a-a346-ca8d4ed5e53d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.595296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3d4bf53a-4661-4833-836d-0b4cb6543beb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.596514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3d4bf53a-4661-4833-836d-0b4cb6543beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.639537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.640220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.640553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.640830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.641554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.643182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.652814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 790/1562] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:54.654122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.658781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 773/1563] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.658781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3d8a-7b82-46d1-9a95-246909ffffc6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.658781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3d8a-7b82-46d1-9a95-246909ffffc6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.658781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3d8a-7b82-46d1-9a95-246909ffffc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.663715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bf8c-d693-48a5-bca0-568a7b9b66e9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.668363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.669872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.670196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.670511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.671234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:54.671544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.673130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.675387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5b6a3d8a-7b82-46d1-9a95-246909ffffc6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.676618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5b6a3d8a-7b82-46d1-9a95-246909ffffc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.684883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 774/1564] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.685625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 791/1565] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:54.686294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.686417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.686683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.686872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.695892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5c0ef-24a6-474f-8c74-3c3cb181cca0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.700737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.702133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.702413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.702686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.703405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:54.703763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.705423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.705552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:54.707143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:54.714988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 775/1566] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:54.716414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.718277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 792/1567] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:54.726973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.727243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc84b-7dae-4a85-9eb7-2e9d5291c527 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.728804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1a64d-9e58-490f-a7d0-d64895c735d2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.732483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.733149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.733431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.733703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.733916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.734461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.734574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.734855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.735124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.735799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.736050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.737679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.746346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 776/1568] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:54.747168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.756385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83ab8-81ba-4e54-82e8-7773d9f32bcb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.760920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.761535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.761820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.762139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.762880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:54.763186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.764746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.776064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 777/1569] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:54.776825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.786454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b77f7-4bb2-4ca1-bb39-bf6025231c53 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.790947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.791783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.792064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.792336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] PATH_INFO: `/v3/projects/7a1ee0cd036149bd9629e1a54287ad2e/users/a9e903b6e7ad4eba8fe9ed75906929c3/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.807254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1ee0cd036149bd9629e1a54287ad2e, user_id=a9e903b6e7ad4eba8fe9ed75906929c3, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.808269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:22:54.808269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:22:54.808269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:22:54.810319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:54.917336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 778/1570] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:54 2022] PUT /identity/v3/projects/7a1ee0cd036149bd9629e1a54287ad2e/users/a9e903b6e7ad4eba8fe9ed75906929c3/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:54.918130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.932242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 793/1571] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:54.933246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:54.947287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b9497-1d10-4b67-8858-967380931873 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.983388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39d23-5c28-48fb-9566-4369e31667a4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:54.988091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:54.990158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:54.990158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:54.990158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:54.990158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:54.991924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.005689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.006406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.006687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.006967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.007729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:55.008033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.009774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.021845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 794/1572] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:55.022771 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.032869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad3013-0598-4c55-b484-6474bd9ad015 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.037591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.038260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.038548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.038827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.039566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:22:55.039872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.041456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.053405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 795/1573] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:22:55.054304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.063747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bde29-4721-4494-8cbd-2094aed3812c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.068468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.069219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.069536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.069819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.070527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.072411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8d8dd-e6e4-494e-b107-f2c12ea42b42 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.077799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 779/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:54 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.077799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.096662 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:55.134698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455bc32-cc5c-4400-9251-2c9fd75ce94f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.139968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.144105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.144105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.144105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.144105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.144105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.153854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 780/1575] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:55.154661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.163721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e11f-4a93-45ac-8b91-247afa2b939a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.168330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.168910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.169193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.169466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.170162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.173538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 796/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.173538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.173767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.173856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.173856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.184463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:55.198503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] MFA Rules not processed for user `a9e903b6e7ad4eba8fe9ed75906929c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:55.199742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:55.263297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 797/1577] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:55.264199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.289290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 781/1578] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.289290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.326852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58790ec6-55bf-4e1a-80e9-e669c2bbc8db None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.351067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8187228-5524-4e43-a3e2-0cddb1d2752c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.355741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.356437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.356722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.357004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.357691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.359666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.382636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.383308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.383598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.383869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.384599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.386234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.396386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 798/1579] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:55.397203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.406663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae09e9-5ea6-4252-be46-ead42494e5b0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.411262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.411879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.412165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.412442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.413130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.415041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.465423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 782/1580] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.465423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.523237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdec7e4-61ae-455c-a63a-b33f56060ff5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.527868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.528553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.528839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.529119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.529798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.531750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.541974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 799/1581] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/projects => generated 391 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.543001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.561835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:55.602410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a4bc-ec3b-4ac6-b12c-ed8ac0455f0b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.606983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.607866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.608148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.608422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] PATH_INFO: `/v3/projects/39849cec37754352a6822cd57277b954/users/0a19c51d287046578e9e4016784dad8f/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.622522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39849cec37754352a6822cd57277b954, user_id=0a19c51d287046578e9e4016784dad8f, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.625379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.627701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 783/1582] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.628337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.638268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5436a6-5ff8-4808-8719-c2d7ef26a2fe admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.642844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.643485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.643769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.644042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.644717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.646667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.654249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:55.712184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 800/1583] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:55 2022] PUT /identity/v3/projects/39849cec37754352a6822cd57277b954/users/0a19c51d287046578e9e4016784dad8f/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:55.713047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.744497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 784/1584] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.745243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.755117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f49be-f81a-4c2b-b9d0-b3b336e872af admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.774774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017108-0ce6-418c-a09f-b133b9c3a42e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.779439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.780115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.780396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.780669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.781386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.783465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.811377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.812083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.812395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.812674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.813372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.815322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.882424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 801/1585] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/projects => generated 369 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.883352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.904825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 785/1586] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:55.905773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.915305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d010b5f-c475-4a58-8ee1-4dfdebb93d8b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.946844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1052322-9ebb-408c-bb06-4b6def07ac56 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.951511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.952224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.952509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.952781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.953526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.955147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.964991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 802/1587] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:55.965773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:55.974320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.975005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.975288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.975517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79a7f4-9dbb-4e87-a2f6-120dbba47c4f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:55.975646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.976269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.978241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:55.980040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:55.980679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:55.980957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:55.981230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:55.982477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:55.984358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493136-b9ec-4c65-931e-3d4555787512 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.058618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 786/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:56.059554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.068981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd05dd-cb5f-41bf-aa38-c2c904cb08c4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.073801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.074471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.074751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.075034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.075751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.077322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.080367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 803/1589] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:55 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:56.080367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.080367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.080367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.086404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:56.094621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 787/1590] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.097653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.097832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:56.099097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:56.105195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f431f7-dd00-4e31-9a48-f87fd674892b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.107697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 804/1591] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:56.108316 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.113619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.114439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.114791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.115138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.115992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.117656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f52b10-4cef-4c7e-bfa1-4f13485104bb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.117919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.122390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.126371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.126371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.126371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.126371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.126371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.128634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 788/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.129492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.140226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd87cd-591d-454e-b369-28f49033307a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.144777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.149336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.149336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.149336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.149336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.149336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.158383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 789/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.159264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.168363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb61a2-4491-4c1f-8c2f-af05f4e9aaee admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.173436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.174289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.178078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.178078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] PATH_INFO: `/v3/projects/f538221717a24ccd9cc4dc31897fbda9/users/77c36c187b804f1da7b65b2655f26fcf/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.188680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f538221717a24ccd9cc4dc31897fbda9, user_id=77c36c187b804f1da7b65b2655f26fcf, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.191320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.232639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 805/1594] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:56 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:56.233572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.242943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6809812-6877-4633-914d-71e90f83054a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.247516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.248144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.248427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.248704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.249444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.251068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.260906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 790/1595] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/f538221717a24ccd9cc4dc31897fbda9/users/77c36c187b804f1da7b65b2655f26fcf/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:56.261782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 806/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.262686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.265416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.275360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0892d-0ea2-4bfb-9156-bf98e9af212f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.279830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:56.328377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ccf7f-379e-4f76-b976-f9824f648ec4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.332075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21500302-daab-401e-b061-742635633697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.332749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500302-daab-401e-b061-742635633697 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.333376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.333524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500302-daab-401e-b061-742635633697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.333623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21500302-daab-401e-b061-742635633697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.333711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.333977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.334283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.334382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21500302-daab-401e-b061-742635633697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.335027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.335950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21500302-daab-401e-b061-742635633697 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.336600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.346600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 807/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.347405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21500302-daab-401e-b061-742635633697 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.350676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 791/1598] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.351374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.362077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e6c87-d7c0-49a6-9d2d-abfe252fcea3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.366571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.367210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.367494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.367770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.368491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.370090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.379945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 792/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.380875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.389982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e575c-517b-4d5d-9128-117d4f43367d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.394554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.395181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.395460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.395730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.396431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.397968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.407971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 793/1600] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.408734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.408857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21500302-daab-401e-b061-742635633697 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.413753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.414422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.414704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.414979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.420671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7d549-dbd2-4d00-b2f4-f1dbd4bd5260 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.424884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.425264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.425905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.426214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.426493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.426731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.427205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.428761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97986d45-218d-44a8-944b-a399c9a41af8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.436257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 808/1601] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:56.437228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.442108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 794/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.442108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.442108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.442108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.447101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dfcbf-57e4-4f54-99cc-ac24eba70727 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.452547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.453319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.453705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.454122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.454947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.456939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.468039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 809/1603] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.475970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.476107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] MFA Rules not processed for user `77c36c187b804f1da7b65b2655f26fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:56.476107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:56.478618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f897e4-0562-466e-b535-c89f2fda9ebe admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.483286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.484148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.484432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.484699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] PATH_INFO: `/v3/projects/cb1d9be9203346cd8cf364305116abf1/users/954933c1cb0249c2a86b309b6f8a334c/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.500953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1d9be9203346cd8cf364305116abf1, user_id=954933c1cb0249c2a86b309b6f8a334c, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.503853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.537366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 795/1604] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:56.538241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.549715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c466b4-94a5-4317-93c3-a022da660e19 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.554398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.555068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.555355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.555637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.556368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.558008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.568079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 796/1605] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:56.568942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.579378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9ba34-0915-4f14-97f5-601a3e3f7d43 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.584305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.585223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.585505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.585782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] PATH_INFO: `/v3/projects/a05af8ceb6d041a6a894d5ca3ff820cc/users/c4586fdc3e8b4ee0ab58d8c236637a4f/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.601115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05af8ceb6d041a6a894d5ca3ff820cc, user_id=c4586fdc3e8b4ee0ab58d8c236637a4f, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.603856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.615440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 810/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/cb1d9be9203346cd8cf364305116abf1/users/954933c1cb0249c2a86b309b6f8a334c/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:56.616388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.649651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:56.676200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25acbed-3741-486b-ae2d-db2840301cff admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.682104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 797/1607] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/a05af8ceb6d041a6a894d5ca3ff820cc/users/c4586fdc3e8b4ee0ab58d8c236637a4f/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:56.682765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.718718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:56.738922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.739865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.740151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.740428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] PATH_INFO: `/v3/projects/3c4adfd8b4c74448bd28b8d0bf149c0a/users/9a58140ac84841dca6ca68a4895abc8a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.749508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bc183-8b99-4cc2-8cec-2b0761b935f9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.756400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.757377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.757665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.758036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] PATH_INFO: `/v3/projects/4cc54f808e554f518bcfa8c2e414fb81/users/817b1169eb7b42e69222262198815051/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.761811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4adfd8b4c74448bd28b8d0bf149c0a, user_id=9a58140ac84841dca6ca68a4895abc8a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.765579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.775124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc54f808e554f518bcfa8c2e414fb81, user_id=817b1169eb7b42e69222262198815051, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.777873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.872964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 798/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/4cc54f808e554f518bcfa8c2e414fb81/users/817b1169eb7b42e69222262198815051/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:56.873905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 811/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/3c4adfd8b4c74448bd28b8d0bf149c0a/users/9a58140ac84841dca6ca68a4895abc8a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:56.874769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.875361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:56.941652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d679d8-4a14-41b0-9741-44d12fb4f9a6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.942982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11144d-268b-4aae-be39-6d806fa9f4da admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:56.947303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.947970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.948223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:56.948384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.948563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.949122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:56.949267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.949445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:56.949680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] PATH_INFO: `/v3/projects/d69eb4f1edc049e9bbca2b0ad51feb26/users/2b655558f6e44de98f1fedb84a4d5839/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:56.951752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f063622-6b9b-4437-a1f3-6c40a3b086d8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:56.968849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69eb4f1edc049e9bbca2b0ad51feb26, user_id=2b655558f6e44de98f1fedb84a4d5839, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:56.972511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.050300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 799/1610] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:56 2022] PUT /identity/v3/projects/d69eb4f1edc049e9bbca2b0ad51feb26/users/2b655558f6e44de98f1fedb84a4d5839/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:57.054734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.054872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 812/1611] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:22:56 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:57.054872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec4bc2-8ae8-40d2-b783-9a37ebf8dfef None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.054872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec4bc2-8ae8-40d2-b783-9a37ebf8dfef None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.054872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec4bc2-8ae8-40d2-b783-9a37ebf8dfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.129139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62184c-ff8b-47f1-836b-af1cd65ad534 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.179585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:57.196908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.197807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.198122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.198848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] PATH_INFO: `/v3/projects/02969314f19c4dc182b0dbe76aad9a34/users/feb66e681a894c148e3463ff5a0dd1be/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.200111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-afec4bc2-8ae8-40d2-b783-9a37ebf8dfef None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.201491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-afec4bc2-8ae8-40d2-b783-9a37ebf8dfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.210104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 813/1612] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.214455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.214743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.215053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.222760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02969314f19c4dc182b0dbe76aad9a34, user_id=feb66e681a894c148e3463ff5a0dd1be, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.226564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fcdc8-1b04-4411-8ff3-ae2beb4fcaa7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.233180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.234420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.242941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 814/1613] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.243607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.254798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67054a-7340-486e-8ca6-a24b8b61c8aa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.259910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.260717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.261059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.261402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.262343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.264168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b74ee-b9a1-4cc1-835e-62f7e55a9dfb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.274527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 815/1614] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:57.276505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.276505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.276505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.293072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.294846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.302451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 816/1615] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.304748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.313015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae4f5-932f-4122-a791-ae7422f14af6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.317732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.318408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.318688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.318961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.319638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.321530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.331825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 800/1616] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:57 2022] PUT /identity/v3/projects/02969314f19c4dc182b0dbe76aad9a34/users/feb66e681a894c148e3463ff5a0dd1be/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:57.334004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.334004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.334004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.361429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] MFA Rules not processed for user `2b655558f6e44de98f1fedb84a4d5839`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.362659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.428410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 801/1617] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.429528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.435726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.446582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:57.454329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 817/1618] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:22:57.454329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.454595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.496397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857458f5-49c2-471b-866f-4a07493683d7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.500657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.501309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.501589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.501860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.512600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b1c47-d9f5-4ae2-b6af-6317ca1cfecc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.516426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.517461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.518059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.518348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.563028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.564465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.574072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 802/1619] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:57.574932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.578201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.579644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.584342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c506d-a0ba-4ce1-979b-861e8652906f service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.589217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.590137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 818/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:57.590283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.590498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.591297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] PATH_INFO: `/v3/projects/4cc54f808e554f518bcfa8c2e414fb81/users/817b1169eb7b42e69222262198815051/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.591486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.593312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc54f808e554f518bcfa8c2e414fb81, user_id=817b1169eb7b42e69222262198815051, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.596035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd6b61-17ec-4311-9b63-d10daa40c76d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.598621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:57.599765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa971c2-81c6-408c-bc80-4b8830747738 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.603695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.604548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.604849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.605107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.625339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 803/1621] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:57 2022] PUT /identity/v3/projects/4cc54f808e554f518bcfa8c2e414fb81/users/817b1169eb7b42e69222262198815051/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:57.625548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.625659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557915b8-4b6d-4cf4-a5d9-8f9e1530f117 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.625659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557915b8-4b6d-4cf4-a5d9-8f9e1530f117 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.625659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557915b8-4b6d-4cf4-a5d9-8f9e1530f117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.625885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbd2c2-d5a9-41a6-a40f-c0dbd7626c73 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.631492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 819/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:57.637682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927156f-accd-4dd2-98b1-80aa964dac73 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.637682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927156f-accd-4dd2-98b1-80aa964dac73 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.637682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927156f-accd-4dd2-98b1-80aa964dac73 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.637682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 820/1623] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 20:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:22:57.637682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927156f-accd-4dd2-98b1-80aa964dac73 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.649252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-557915b8-4b6d-4cf4-a5d9-8f9e1530f117 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.650589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-557915b8-4b6d-4cf4-a5d9-8f9e1530f117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.702770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927156f-accd-4dd2-98b1-80aa964dac73 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.707938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.709073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.709474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.709751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.718426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.724415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 804/1624] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.727570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758815c-873a-4796-96b9-354bd41c1550 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.728715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.729155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.729448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.747596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 821/1625] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:57.755236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.755606 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.755958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.760865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] MFA Rules not processed for user `817b1169eb7b42e69222262198815051`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.762166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.776466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:57.777838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:57.837307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 805/1626] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.838364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.852650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eea32e-abf0-4998-a7dd-5b32c3d24b78 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.855171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 822/1627] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:57.856175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.859157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.860601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.860968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.861341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] PATH_INFO: `/v3/projects/57c9bb1dc6d24274b08595aa8269961c/users/55a8f3839ec54c9aabd22d92cf8a579d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.888633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c9bb1dc6d24274b08595aa8269961c, user_id=55a8f3839ec54c9aabd22d92cf8a579d, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.891469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.926737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288e0c3-d224-4531-a7e7-a7db5638d6d8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:57.930382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:57.931054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:57.931337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:57.931612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:57.941676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:57.943222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:57.954364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 823/1628] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:57.955204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:57.975796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 806/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:57 2022] PUT /identity/v3/projects/57c9bb1dc6d24274b08595aa8269961c/users/55a8f3839ec54c9aabd22d92cf8a579d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:57.978848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:58.024498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:58.063103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0a262-9e8f-439c-b065-e9db3c8a4140 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.087216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a990f9d-28e5-4aa6-ab74-ba700e107be0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.094586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.095254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.095542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.095819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.116620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.117281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.117564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.117836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.156736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.158922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.181741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 807/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:58.184991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:58.194786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.197347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.239042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 824/1631] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:58.300280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b82-b92c-44be-85df-333ff7e824a9 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.305689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.306595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.306983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.307930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.308784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.311344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283a353-5be5-4a34-9f52-2789e030957d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.333043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 808/1632] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:22:58.334359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:58.346500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09cf1-fdf1-426a-9ec5-51c7fa01ca3f service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.353111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.354250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.354697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.355181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] PATH_INFO: `/v3/projects/57c9bb1dc6d24274b08595aa8269961c/users/55a8f3839ec54c9aabd22d92cf8a579d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.358387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c9bb1dc6d24274b08595aa8269961c, user_id=55a8f3839ec54c9aabd22d92cf8a579d, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.362210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.404158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 825/1633] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:22:58 2022] PUT /identity/v3/projects/57c9bb1dc6d24274b08595aa8269961c/users/55a8f3839ec54c9aabd22d92cf8a579d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:22:58.405312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1a2a-a328-4972-84e9-f66459f455af None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.405731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1a2a-a328-4972-84e9-f66459f455af None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.406447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1a2a-a328-4972-84e9-f66459f455af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.419201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:58.463213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-de7d1a2a-a328-4972-84e9-f66459f455af None None] MFA Rules not processed for user `55a8f3839ec54c9aabd22d92cf8a579d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:58.464557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-de7d1a2a-a328-4972-84e9-f66459f455af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:58.509709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:22:58.628109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 809/1634] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:22:58 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:58.640805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:58.650556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3ca83-726e-4ef0-9859-c3cee089b341 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.650855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3ca83-726e-4ef0-9859-c3cee089b341 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.651465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3ca83-726e-4ef0-9859-c3cee089b341 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.652980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 810/1635] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 20:22:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:22:58.669433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbcd-10b7-4635-b393-e37104bad5c4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.669892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbcd-10b7-4635-b393-e37104bad5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.670360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8cbcd-10b7-4635-b393-e37104bad5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.706628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d9e8cbcd-10b7-4635-b393-e37104bad5c4 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:22:58.707862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d9e8cbcd-10b7-4635-b393-e37104bad5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:22:58.713617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bf3ad-8811-4c98-b1ea-86b813354699 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.717646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.718408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.718694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.719058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.737120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.738803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.750945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 826/1636] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:58.803370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 811/1637] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:22:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:22:58.892778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:22:58.902274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655e5cb-286b-4c4d-b593-56bb22b26e9d service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:22:58.918303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:22:58.920396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:22:58.920396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:22:58.920661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:22:58.967508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:22:58.968389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729741d-f25e-40e8-af15-618059369e6d service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:22:58.999812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 827/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:22:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:22:59.350293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:00.149686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:00.150878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:00.151227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:00.267787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] MFA Rules not processed for user `9a58140ac84841dca6ca68a4895abc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:00.269065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:00.718681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 812/1639] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65c5c8-db7a-4686-bc56-0da9b9b8ce5f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65c5c8-db7a-4686-bc56-0da9b9b8ce5f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65c5c8-db7a-4686-bc56-0da9b9b8ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3e65c5c8-db7a-4686-bc56-0da9b9b8ce5f None None] MFA Rules not processed for user `9a58140ac84841dca6ca68a4895abc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3e65c5c8-db7a-4686-bc56-0da9b9b8ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:00.755957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 828/1640] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:00.760613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:00.760613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006c70c-eeb8-4334-892d-65fecbe6c78f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:00.760613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:00.760613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:00.763923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:00.763923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:00.763923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:00.763923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162fdc0-7b9d-4030-b4f3-cec53f1262f9 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:00.763923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 813/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:00.968195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f86dd1-07c8-4626-8cab-0cd520271849 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:00.968727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f86dd1-07c8-4626-8cab-0cd520271849 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:00.969187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f86dd1-07c8-4626-8cab-0cd520271849 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:00.970932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 829/1642] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 20:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:00.979474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdf2-7f2f-465c-9b77-eda56e840336 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:00.979975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdf2-7f2f-465c-9b77-eda56e840336 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:00.980438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bdf2-7f2f-465c-9b77-eda56e840336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.018234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4ad9bdf2-7f2f-465c-9b77-eda56e840336 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:01.019923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4ad9bdf2-7f2f-465c-9b77-eda56e840336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:01.035583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 814/1643] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 20:23:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:01.042414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bede3e3-257b-49d9-9a56-4c859055c109 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:01.043059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bede3e3-257b-49d9-9a56-4c859055c109 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:01.043516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bede3e3-257b-49d9-9a56-4c859055c109 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.046695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 830/1644] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 20:23:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:01.053805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:01.057328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:01.057328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.092527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:01.103359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:01.126074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 815/1645] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:01.149236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bede3e3-257b-49d9-9a56-4c859055c109 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:01.151745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:01.169736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acd17b-9f6c-4901-bd9c-a7651c814013 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:01.176485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:01.177497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:01.178467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:01.179325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.207187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:01.214542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897f793-e99a-4a4f-b0c6-f59912c3134e service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:01.258117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 816/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:01.299798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bede3e3-257b-49d9-9a56-4c859055c109 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:01.308125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:01.309340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:01.309789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:01.310234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] PATH_INFO: `/v3/users/77c36c187b804f1da7b65b2655f26fcf` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.311949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c36c187b804f1da7b65b2655f26fcf)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:01.314600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:01.521058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:01.526272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c36c187b804f1da7b65b2655f26fcf'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:01.527238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c36c187b804f1da7b65b2655f26fcf'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:01.528834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c36c187b804f1da7b65b2655f26fcf'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:01.553982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c36c187b804f1da7b65b2655f26fcf'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:01.564699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c36c187b804f1da7b65b2655f26fcf'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:01.692486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 831/1647] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:01 2022] DELETE /identity/v3/users/77c36c187b804f1da7b65b2655f26fcf => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:01.694802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25532d7-0afc-4940-b3f4-ba128f6b1d11 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:01.695328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25532d7-0afc-4940-b3f4-ba128f6b1d11 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:01.695799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25532d7-0afc-4940-b3f4-ba128f6b1d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:01.791576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d25532d7-0afc-4940-b3f4-ba128f6b1d11 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:01.794923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d25532d7-0afc-4940-b3f4-ba128f6b1d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:01.909020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:01.935809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 817/1648] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:01.946189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:01.947901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:02.096846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3543c-8289-4c94-a4a2-5468c9bcac80 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:02.114100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:02.117558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:02.117558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:02.117558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:02.135707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:02.137192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa96101-59fe-4f51-85ca-75ad1d6d3f17 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:02.164739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 832/1649] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:02.349462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:02.349903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:02.350034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:02.357846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 818/1650] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 20:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:02.358937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:02.359624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:02.359913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:02.444732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:02.446154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:02.465222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 833/1651] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:02.553208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:02.558188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:02.587397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe21c9f-4daf-492a-9723-4c3242baeb5b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:02.598895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:02.603389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:02.603862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:02.604306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:02.622648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:02.627687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:02.647912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 819/1652] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:03.136923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:03.139657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:03.167376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef846a6-7cf8-4fdb-a00d-43e13aaacc0a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:03.181710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:03.182674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:03.182963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:03.183254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] PATH_INFO: `/v3/projects/f538221717a24ccd9cc4dc31897fbda9` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:03.185145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f538221717a24ccd9cc4dc31897fbda9)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:03.186247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:23:03.186247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 23 20:23:03.186247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: May 23 20:23:03.187479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:03.373744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f538221717a24ccd9cc4dc31897fbda9', 'request_id': 'req-452ad23e-0468-446b-bd00-a09865a8573e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.469209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:03.472022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:03.491741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabb90c-7e41-4cf6-abb6-618ef6f8c843 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:03.500063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f538221717a24ccd9cc4dc31897fbda9', 'request_id': 'req-452ad23e-0468-446b-bd00-a09865a8573e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.550888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f538221717a24ccd9cc4dc31897fbda9', 'request_id': 'req-452ad23e-0468-446b-bd00-a09865a8573e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.575056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:03.576293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:03.576697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:03.577044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] PATH_INFO: `/v3/users/a9e903b6e7ad4eba8fe9ed75906929c3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:03.580401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e903b6e7ad4eba8fe9ed75906929c3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:03.582744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 20:23:03.582744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 23 20:23:03.582744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: May 23 20:23:03.585148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:03.588420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] The token cache is being invalidate because project f538221717a24ccd9cc4dc31897fbda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:03.588732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.591948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.598517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 834/1653] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:03 2022] DELETE /identity/v3/projects/f538221717a24ccd9cc4dc31897fbda9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:03.599456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:03.602662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:03.645564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ad23e-0468-446b-bd00-a09865a8573e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:03.667445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:03.668402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:03.669320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:03.669681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] PATH_INFO: `/v3/users/55a8f3839ec54c9aabd22d92cf8a579d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:03.673308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a8f3839ec54c9aabd22d92cf8a579d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:03.680280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:03.901091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e903b6e7ad4eba8fe9ed75906929c3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.905741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e903b6e7ad4eba8fe9ed75906929c3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.906723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e903b6e7ad4eba8fe9ed75906929c3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.945416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e903b6e7ad4eba8fe9ed75906929c3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.947739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a8f3839ec54c9aabd22d92cf8a579d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.948829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a8f3839ec54c9aabd22d92cf8a579d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.950337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a8f3839ec54c9aabd22d92cf8a579d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.970536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a8f3839ec54c9aabd22d92cf8a579d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.979298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e903b6e7ad4eba8fe9ed75906929c3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.980365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b5f788c4-ac47-4a14-8053-ae25b1d3ccbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a8f3839ec54c9aabd22d92cf8a579d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:03.996177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:04.072918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 835/1654] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:03 2022] DELETE /identity/v3/users/55a8f3839ec54c9aabd22d92cf8a579d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:04.130136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 820/1655] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:03 2022] DELETE /identity/v3/users/a9e903b6e7ad4eba8fe9ed75906929c3 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:04.135916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdb450-4bb0-4754-bde7-68758d5d5148 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:04.136862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdb450-4bb0-4754-bde7-68758d5d5148 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:04.137893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdb450-4bb0-4754-bde7-68758d5d5148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:04.185586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9ecdb450-4bb0-4754-bde7-68758d5d5148 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:04.188466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9ecdb450-4bb0-4754-bde7-68758d5d5148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:04.322423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 836/1656] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:04.353335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:04.356732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:04.632787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883258-eb19-4062-b426-c540a3a1a2e7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:04.647283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:04.647975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:04.651553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:04.651553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:04.681377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:04.687956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4748a0-9251-415e-9eff-88df379affcb service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:04.717243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 821/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:05.164805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163bd77-ea06-4407-a92e-54c9a3c7583d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:05.165308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163bd77-ea06-4407-a92e-54c9a3c7583d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:05.165394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163bd77-ea06-4407-a92e-54c9a3c7583d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:05.167475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 837/1658] 10.222.0.16 () {54 vars in 960 bytes} [Mon May 23 20:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:05.186726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:05.186726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:05.186726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:05.189015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:05.219209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:05.220678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:05.364482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 822/1659] 10.222.0.16 () {58 vars in 1057 bytes} [Mon May 23 20:23:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:05.375805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163bd77-ea06-4407-a92e-54c9a3c7583d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:05.414702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163bd77-ea06-4407-a92e-54c9a3c7583d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:05.433523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:05.434863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:05.435155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:05.435432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:05.446064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:05.471776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:05.473414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd18a00-21b8-48f3-b717-f2790f626d27 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:05.502585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 838/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:05.648112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:05.650352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:05.675225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c2a74-9768-427e-b86b-ca84533bc5fe None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:05.694277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:05.695774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:05.697706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:05.698324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] PATH_INFO: `/v3/projects/57c9bb1dc6d24274b08595aa8269961c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:05.703730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c9bb1dc6d24274b08595aa8269961c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:05.717318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:05.964237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c9bb1dc6d24274b08595aa8269961c', 'request_id': 'req-f8ee4ae7-910e-4395-bfbd-e405bf12d898'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:05.964365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:05.964569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:05.964855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.039872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c9bb1dc6d24274b08595aa8269961c', 'request_id': 'req-f8ee4ae7-910e-4395-bfbd-e405bf12d898'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:06.048843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c9bb1dc6d24274b08595aa8269961c', 'request_id': 'req-f8ee4ae7-910e-4395-bfbd-e405bf12d898'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:06.060952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] MFA Rules not processed for user `0a19c51d287046578e9e4016784dad8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:06.070770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:06.089526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] The token cache is being invalidate because project 57c9bb1dc6d24274b08595aa8269961c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:06.089918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:06.093227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f8ee4ae7-910e-4395-bfbd-e405bf12d898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:06.098200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 823/1661] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:05 2022] DELETE /identity/v3/projects/57c9bb1dc6d24274b08595aa8269961c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:06.099810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf6f68-ae23-4078-b933-459c11199f00 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.100303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf6f68-ae23-4078-b933-459c11199f00 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.100727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf6f68-ae23-4078-b933-459c11199f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.139773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7adf6f68-ae23-4078-b933-459c11199f00 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:06.144152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7adf6f68-ae23-4078-b933-459c11199f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:06.249385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 839/1662] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:06.253067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:06.255876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:06.284584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:06.357347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 824/1663] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:06.361232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.364392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.364698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.386925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391326-8d78-4f03-8d08-633bd380aab0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:06.394972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:06.395656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.413488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:06.415041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.416522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.417873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:06.440797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 825/1664] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:06.444113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:06.444113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:06.445379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:06.469715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeb0c4-1590-424e-9ba5-bfc50a4ea0c0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:06.483994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:06.484755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.485109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.485461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.486299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:06.486681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:06.491072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:06.516184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 840/1665] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:06.525099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 826/1666] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:06.529861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:06.553494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb72cd-c69c-4701-b533-245968b3a685 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:06.568470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:06.569225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.569494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.569783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.570536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:06.572472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:06.723623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 841/1667] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:06 2022] POST /identity/v3/projects => generated 355 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:06.724107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:06.744393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:06.753738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9f140-078c-464d-b09b-33274eecd07a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:06.864679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:06.866503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:06.943854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:06.945595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:06.948305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:06.948932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:06.950985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:06.954514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.077679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 827/1668] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:06 2022] POST /identity/v3/users => generated 278 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:07.080590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:07.118345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31a647-f989-42f6-8f27-05165610ff6a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:07.127890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:07.130200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b767a-588c-411e-93ef-9cf9481cd9b7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:07.132310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.132785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.133229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] PATH_INFO: `/v3/projects/7a1ee0cd036149bd9629e1a54287ad2e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.134838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1ee0cd036149bd9629e1a54287ad2e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:07.137077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.156423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:07.157061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.157347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.157630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.158707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:07.160206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.182875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 828/1669] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:07.187273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:07.208556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e7f9-3a23-422e-baa7-df0058f80892 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:07.231422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:07.232048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.232336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.232616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.233363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:07.239907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.287315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 829/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:23:07.293041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:07.313503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1ee0cd036149bd9629e1a54287ad2e', 'request_id': 'req-d8d8eab7-2734-4544-b024-08f682b66dee'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.318462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf6790-0ea7-417b-8526-d6b5d452ed20 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:07.409342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1ee0cd036149bd9629e1a54287ad2e', 'request_id': 'req-d8d8eab7-2734-4544-b024-08f682b66dee'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.424905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1ee0cd036149bd9629e1a54287ad2e', 'request_id': 'req-d8d8eab7-2734-4544-b024-08f682b66dee'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.449080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] The token cache is being invalidate because project 7a1ee0cd036149bd9629e1a54287ad2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:07.449428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.450516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.455667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 842/1671] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:06 2022] DELETE /identity/v3/projects/7a1ee0cd036149bd9629e1a54287ad2e => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:07.460808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:07.462766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:07.479747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:07.480693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.481576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.481858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] PATH_INFO: `/v3/projects/5642c4590d4c440fb835a76ba9f0869a/users/f62a3b686048478ea09c53c35f9c9921/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.515551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=5642c4590d4c440fb835a76ba9f0869a, user_id=f62a3b686048478ea09c53c35f9c9921, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:07.518311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116e8c6-9370-4407-b1ca-1ba26477f8ea admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.576075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8eab7-2734-4544-b024-08f682b66dee admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:07.583506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:07.584896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.585366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.585796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] PATH_INFO: `/v3/users/817b1169eb7b42e69222262198815051` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.587544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] RBAC: Authorizing `identity:delete_user(user_id=817b1169eb7b42e69222262198815051)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:07.590083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:07.619602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 830/1672] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:07 2022] PUT /identity/v3/projects/5642c4590d4c440fb835a76ba9f0869a/users/f62a3b686048478ea09c53c35f9c9921/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:07.631946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:07.634754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:07.635510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:07.683398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] MFA Rules not processed for user `f62a3b686048478ea09c53c35f9c9921`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:07.685447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:07.868139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817b1169eb7b42e69222262198815051'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.869419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817b1169eb7b42e69222262198815051'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.872971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817b1169eb7b42e69222262198815051'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.878968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 831/1673] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:07 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:07.889497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:07.890586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817b1169eb7b42e69222262198815051'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:07.896115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:07.901354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817b1169eb7b42e69222262198815051'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:08.010254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 843/1674] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:07 2022] DELETE /identity/v3/users/817b1169eb7b42e69222262198815051 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:08.065731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3eadf-2f81-435f-beb3-f5785d0386c1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:08.078380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:08.159786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:08.160475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:08.160760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:08.161047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:08.161856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:08.165992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:08.264982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 832/1675] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:07 2022] POST /identity/v3/projects => generated 357 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:08.268615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:08.270419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:08.286744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdbbd-22eb-488e-aad5-19a0b03451db admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:08.571248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:08.572642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:08.573384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:08.573822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:08.576114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:08.579137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:08.733739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 844/1676] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:08 2022] POST /identity/v3/users => generated 285 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:08.735603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:08.762803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1485b-c575-4062-b15d-e990cbb3946e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:08.768114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:08.769005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:08.769431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:08.769845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:08.770892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:08.772832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:08.808364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 833/1677] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:23:08.812920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:08.829691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a2fe-7c25-40a3-bc75-dfc0948fa150 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:08.836871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:08.837949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:08.838548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:08.839004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:08.839568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] PATH_INFO: `/v3/projects/32312c5c1cc34f2199d8b84003f6a112/users/2d01f1c4113a495a90b1e627eb48b1cb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:08.862069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=32312c5c1cc34f2199d8b84003f6a112, user_id=2d01f1c4113a495a90b1e627eb48b1cb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:08.865121 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:08.936104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 845/1678] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:08 2022] PUT /identity/v3/projects/32312c5c1cc34f2199d8b84003f6a112/users/2d01f1c4113a495a90b1e627eb48b1cb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:08.943498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:08.955445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f0e2-7211-4d8b-9b17-920b0cf5cb64 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:09.054884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:09.055768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.056170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.056678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.057862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:09.060892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77e469-2716-4dae-9214-afb8bedaaa8e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:09.074079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 834/1679] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:09.079390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:09.091492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd73db-f101-4703-907b-4d5ce2e62cfa admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:09.096543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:09.097892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.098345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.098788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] PATH_INFO: `/v3/projects/32312c5c1cc34f2199d8b84003f6a112/users/2d01f1c4113a495a90b1e627eb48b1cb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.100961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=32312c5c1cc34f2199d8b84003f6a112, user_id=2d01f1c4113a495a90b1e627eb48b1cb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:09.103973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:09.117153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 846/1680] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:09 2022] PUT /identity/v3/projects/32312c5c1cc34f2199d8b84003f6a112/users/2d01f1c4113a495a90b1e627eb48b1cb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:09.128857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.129238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.129597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.162533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] MFA Rules not processed for user `2d01f1c4113a495a90b1e627eb48b1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:09.163965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:09.258276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 835/1681] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:09 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:09.267130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:09.269310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:09.353959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98704c0-5d9b-4ad2-af00-a61322e62fbc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:09.358700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:09.359886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.360257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.360608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.373270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:09.374790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:09.411870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 847/1682] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:09 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:09.619206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:09.626318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:09.702752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:09.714563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e7ca0-a030-4979-a421-7326af2b2f8f service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:09.740203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:09.741019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.743002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.745994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.753831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:09.879321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:09.883933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922398c0-de53-48fc-8b9f-a68f3b450d39 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:09.901127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:09.906407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:09.906694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:09.906976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] PATH_INFO: `/v3/projects/4cc54f808e554f518bcfa8c2e414fb81` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:09.913323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc54f808e554f518bcfa8c2e414fb81)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:09.922528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:09.937694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:09.939562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:09.952589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 848/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:09 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:10.004074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:10.144557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:10.146047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:10.171573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44c8c3-41d0-469b-aa80-6682c1c913fd service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:10.187169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:10.189364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.203805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.204283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] PATH_INFO: `/v3/users/2b655558f6e44de98f1fedb84a4d5839` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.217615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b655558f6e44de98f1fedb84a4d5839)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:10.224730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:10.231303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc54f808e554f518bcfa8c2e414fb81', 'request_id': 'req-78b5603b-1412-4686-8ddd-06ff047a5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.351920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc54f808e554f518bcfa8c2e414fb81', 'request_id': 'req-78b5603b-1412-4686-8ddd-06ff047a5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.366939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc54f808e554f518bcfa8c2e414fb81', 'request_id': 'req-78b5603b-1412-4686-8ddd-06ff047a5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.405343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] The token cache is being invalidate because project 4cc54f808e554f518bcfa8c2e414fb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:10.405733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.407484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-78b5603b-1412-4686-8ddd-06ff047a5e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.409760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 836/1684] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:09 2022] DELETE /identity/v3/projects/4cc54f808e554f518bcfa8c2e414fb81 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:10.420165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a471e6-98f3-4a45-9ef4-8088e14cd57c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.420455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a471e6-98f3-4a45-9ef4-8088e14cd57c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.422877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a471e6-98f3-4a45-9ef4-8088e14cd57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.501405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b655558f6e44de98f1fedb84a4d5839'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.504800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b655558f6e44de98f1fedb84a4d5839'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.505989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b655558f6e44de98f1fedb84a4d5839'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.517079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a2a471e6-98f3-4a45-9ef4-8088e14cd57c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:10.521066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a2a471e6-98f3-4a45-9ef4-8088e14cd57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:10.548787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b655558f6e44de98f1fedb84a4d5839'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.574314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a305b3dd-ea13-4a44-b20b-66712a497bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b655558f6e44de98f1fedb84a4d5839'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:10.623017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 837/1685] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:10.630770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.631520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.633379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.659478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:10.661389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:10.674747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 838/1686] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:10.678904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:10.694057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ab8b1-5b50-4c2c-9ea4-8421d25997b6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:10.715702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 849/1687] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:10 2022] DELETE /identity/v3/users/2b655558f6e44de98f1fedb84a4d5839 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:10.720767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:10.721068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.721996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.722311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.724337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.725926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.726580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.729697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:10.749215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:10.753771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:10.784349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:10.785808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:10.812956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 839/1688] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:10.818067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:10.837575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e1c6-51a8-4b38-81e5-f9a6a2119e25 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:10.950405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 850/1689] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:10.981067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:10.982699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:10.984067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:10.984926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:10.985383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:10.986146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:10.987016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:10.989736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1136a-da14-4a3e-9a59-c617a581cae9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.103762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693aeac-3ac5-49da-85fb-26b7fbe060ad None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:11.108614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137350-625f-4924-bde1-853b967315ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:11.109496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137350-625f-4924-bde1-853b967315ec service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.109957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137350-625f-4924-bde1-853b967315ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.110455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137350-625f-4924-bde1-853b967315ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.131622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137350-625f-4924-bde1-853b967315ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:11.133219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 840/1690] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:10 2022] POST /identity/v3/projects => generated 369 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:11.135960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.136076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137350-625f-4924-bde1-853b967315ec service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.138220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.138697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.163684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 851/1691] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:11.166418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137350-625f-4924-bde1-853b967315ec service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:11.189810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:11.191374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:11.195086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137350-625f-4924-bde1-853b967315ec service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:11.332482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 841/1692] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:23:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:11.357722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:11.358384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.358669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.358939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.359893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:11.362041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.386397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:11.548116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 852/1693] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:11 2022] POST /identity/v3/users => generated 285 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:11.552040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:11.571931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08c201-ffe7-46f3-b1e1-14754fb829e1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:11.577007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:11.577705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.577996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.578329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.579084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:11.581095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.597786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 842/1694] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:11.601827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:11.613898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f824f-a955-4b05-bf88-6bc120f94afa admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:11.632429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:11.633124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.633410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.633690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.634460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:11.636100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c99cc-0ff1-420d-b592-4812b9413169 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.659677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:11.666456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 853/1695] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:11.671822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:11.686446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29111a6-064b-4e05-85cb-3a3d9a6ff7f7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:11.703032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:11.704071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.704457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.704830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] PATH_INFO: `/v3/projects/ee69177d6eed4ea390393c316e107099/users/fc6ba6fb5a7a431c95484263d02efb63/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.742534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee69177d6eed4ea390393c316e107099, user_id=fc6ba6fb5a7a431c95484263d02efb63, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:11.753621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:11.833691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 843/1696] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:11 2022] PUT /identity/v3/projects/ee69177d6eed4ea390393c316e107099/users/fc6ba6fb5a7a431c95484263d02efb63/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:11.844697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:11.845189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:11.845277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:11.848197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:11.854879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:11.934170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] MFA Rules not processed for user `fc6ba6fb5a7a431c95484263d02efb63`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:11.934170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:12.038916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f1ea-acec-4159-abee-884e42f30e4c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:12.050532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:12.052176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:12.053181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:12.054703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:12.081142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 854/1697] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:12.083691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:12.171587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:12.176938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:12.235094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 844/1698] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:23:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:23:12.289154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0714-7d04-4b25-a93c-f5ad3d0339d7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:12.302497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:12.303937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:12.304442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:12.305011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:12.306095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:12.311214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:12.451369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:12.453550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:12.477941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 855/1699] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:12 2022] POST /identity/v3/projects => generated 367 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:12.482209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:12.533943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0960c0-d800-4c00-855d-46cc22deebb9 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:12.844724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba555e-8974-428b-99cc-ac11e0505d70 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:12.853381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:12.854527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:12.858573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:12.859870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] PATH_INFO: `/v3/projects/d69eb4f1edc049e9bbca2b0ad51feb26` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:12.868163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:12.869007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:12.870776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:12.870776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:12.872563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:12.873326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69eb4f1edc049e9bbca2b0ad51feb26)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:12.875384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:12.886632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:13.015015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 856/1700] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:12 2022] POST /identity/v3/users => generated 290 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:13.019048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:13.046898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a018be-d4ce-4e8f-9e88-730518b26b31 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:13.073640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:13.076610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:13.079078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:13.084406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:13.086237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:13.088077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:13.125970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69eb4f1edc049e9bbca2b0ad51feb26', 'request_id': 'req-01249405-8b0b-43a5-869f-32ecb1509830'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:13.134516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 857/1701] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:13.140510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:13.182228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1f435-f9be-45ea-88cb-64893fcf527e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:13.266062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69eb4f1edc049e9bbca2b0ad51feb26', 'request_id': 'req-01249405-8b0b-43a5-869f-32ecb1509830'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:13.274926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69eb4f1edc049e9bbca2b0ad51feb26', 'request_id': 'req-01249405-8b0b-43a5-869f-32ecb1509830'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:13.311828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] The token cache is being invalidate because project d69eb4f1edc049e9bbca2b0ad51feb26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:13.314411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:13.318265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-01249405-8b0b-43a5-869f-32ecb1509830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:13.330255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 845/1702] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:12 2022] DELETE /identity/v3/projects/d69eb4f1edc049e9bbca2b0ad51feb26 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:13.335382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3843a-88b4-4ca3-94b6-f39ad0a051ad None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:13.335869 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3843a-88b4-4ca3-94b6-f39ad0a051ad None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:13.336308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3843a-88b4-4ca3-94b6-f39ad0a051ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:13.340973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:13.441066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0ac3843a-88b4-4ca3-94b6-f39ad0a051ad None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:13.441066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0ac3843a-88b4-4ca3-94b6-f39ad0a051ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:13.488550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:13.490044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:13.490550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:13.491225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] PATH_INFO: `/v3/projects/468f6516a2ef44c8be8edf1842e47907/users/279daf7bcbdf4f7283f14ddf48129fcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:13.512787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] RBAC: Authorizing `identity:create_grant(project_id=468f6516a2ef44c8be8edf1842e47907, user_id=279daf7bcbdf4f7283f14ddf48129fcb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:13.523545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:13.527858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:13.833077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 846/1703] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:13.843137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e134bd-a6c2-43c1-97b5-acfc7be973ab None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:13.843684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e134bd-a6c2-43c1-97b5-acfc7be973ab None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:13.844177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e134bd-a6c2-43c1-97b5-acfc7be973ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:13.928942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-31e134bd-a6c2-43c1-97b5-acfc7be973ab None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:13.937212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-31e134bd-a6c2-43c1-97b5-acfc7be973ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:13.987838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 847/1704] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:13.989870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:13.990358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:13.990894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.070083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] MFA Rules not processed for user `954933c1cb0249c2a86b309b6f8a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:14.072222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:14.099492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 858/1705] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:13 2022] PUT /identity/v3/projects/468f6516a2ef44c8be8edf1842e47907/users/279daf7bcbdf4f7283f14ddf48129fcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:14.100522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:14.302102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 848/1706] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:14.304261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:14.312242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:14.343485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645b79a-d7c4-43e6-9b17-8fac80e9605d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:14.366936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:14.370462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.370462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.370462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.370462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:14.370768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96a37-b43f-4369-9fd8-466b9f91822d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:14.372010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:14.384604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:14.387040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.388287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.388833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.392697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:14.393259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:14.399912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 849/1707] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:14.399912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:14.406032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:14.406881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:14.444948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 859/1708] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:14.447701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:14.474449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21057df0-eb0c-415c-845d-c80b704f311a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:14.486537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:14.490676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.495016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.495016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] PATH_INFO: `/v3/projects/468f6516a2ef44c8be8edf1842e47907/users/279daf7bcbdf4f7283f14ddf48129fcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.503666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468f6516a2ef44c8be8edf1842e47907, user_id=279daf7bcbdf4f7283f14ddf48129fcb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:14.524386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:14.597325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 860/1709] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:14 2022] PUT /identity/v3/projects/468f6516a2ef44c8be8edf1842e47907/users/279daf7bcbdf4f7283f14ddf48129fcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:14.600474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:14.658212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aafabf-60db-426d-b7d4-5a4723c88c65 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:14.803887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:14.816291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651847e-6d83-4674-9cee-56fa1a30a735 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:14.829099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:14.829954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.832420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.832801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.834470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:14.839326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f5aa3-fc06-499b-aeb5-32e183dc0829 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:14.863624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:14.864380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.864774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.865153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.979200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 861/1710] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:14 2022] POST /identity/v3/projects => generated 365 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:14.983974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975828bf-4d62-40a1-b355-802cee121479 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:14.985096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975828bf-4d62-40a1-b355-802cee121479 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:14.985881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975828bf-4d62-40a1-b355-802cee121479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:14.990236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:14.992195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c82d0f-21a6-4f25-b6fa-7e472f1a5b52 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:15.025077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:15.040036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-975828bf-4d62-40a1-b355-802cee121479 None None] MFA Rules not processed for user `0a19c51d287046578e9e4016784dad8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:15.041777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-975828bf-4d62-40a1-b355-802cee121479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:15.050635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 850/1711] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:15.054816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.055166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.056002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.162707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] MFA Rules not processed for user `279daf7bcbdf4f7283f14ddf48129fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:15.164362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:15.248169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 862/1712] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:15.252782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975828bf-4d62-40a1-b355-802cee121479 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.392620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975828bf-4d62-40a1-b355-802cee121479 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.456842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.456842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.457134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.457522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.458191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:15.460384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:15.475833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 851/1713] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:15.501054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.502832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:15.603188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 863/1714] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:15 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:15.607971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.661398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2a1d6-6933-4928-8a53-8a194a364277 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.683109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.683864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.684321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.684516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.685233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:15.696997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb1431-8dc6-4b6b-9d34-3810a68555be None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.701281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.702962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.703412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.703932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.710271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:15.725354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:15.729387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:15.762285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 864/1715] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:15.766159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.778547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 852/1716] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:15.812194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002359e-b743-4276-a9b2-626e166c0107 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.842782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.843490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.843780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.844062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.844882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:15.846574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:15.885475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 865/1717] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:15.890407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.916315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d372b-4bdb-4903-a8e4-e21ac52fd9ec service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.936299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:15.940343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:15.954796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.956253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.957901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.958445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] PATH_INFO: `/v3/projects/40888437d23e4f86958b7e7c5cdee27e/users/435ccc836ece4a4193cd0a89e7426667/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.973274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481edd19-2d16-49cd-8a17-c133cc8ea5ea admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:15.985500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:15.986714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:15.992091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:15.992719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:15.998985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:16.022367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40888437d23e4f86958b7e7c5cdee27e, user_id=435ccc836ece4a4193cd0a89e7426667, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:16.030149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a9aa-ded5-4f1c-9b16-1bc53ceed1e6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:16.133423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 853/1718] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:15 2022] PUT /identity/v3/projects/40888437d23e4f86958b7e7c5cdee27e/users/435ccc836ece4a4193cd0a89e7426667/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:16.145380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af1ce-c111-44fc-8a08-684b5df9dd40 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:16.145860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af1ce-c111-44fc-8a08-684b5df9dd40 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:16.146183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849af1ce-c111-44fc-8a08-684b5df9dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:16.173938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:16.179909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:16.211940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 866/1719] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:16.232451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-849af1ce-c111-44fc-8a08-684b5df9dd40 None None] MFA Rules not processed for user `435ccc836ece4a4193cd0a89e7426667`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:16.235356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-849af1ce-c111-44fc-8a08-684b5df9dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:16.255127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:16.414089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 854/1720] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:16.425325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:16.435075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf9d4-dc78-45d4-ac6a-f23db0696410 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:16.567189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:16.569721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:16.570283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:16.570755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:16.599006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:16.600583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27ea1e-bd31-4c76-ad23-7b389d5b3b7b service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:16.637758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 867/1721] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:16.672385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:16.741778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:16.741778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:16.741778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:16.743831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 855/1722] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 20:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:16.789866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a3269-3cf0-416f-9c5c-0cd8ca0256c9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:16.790188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a3269-3cf0-416f-9c5c-0cd8ca0256c9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:16.790472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a3269-3cf0-416f-9c5c-0cd8ca0256c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:16.865222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-581a3269-3cf0-416f-9c5c-0cd8ca0256c9 None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:16.867400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-581a3269-3cf0-416f-9c5c-0cd8ca0256c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:17.000752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 868/1723] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:23:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:17.042907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:17.045518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:17.071304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcca0-93b5-4e52-b15e-66d384bafeb0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:17.077084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:17.082695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:17.082695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:17.082695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:17.093723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:17.096186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:17.117857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 856/1724] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:17.452706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1702-c7f5-460e-9580-77bd2d2a69fc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:17.457590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1702-c7f5-460e-9580-77bd2d2a69fc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:17.458231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1702-c7f5-460e-9580-77bd2d2a69fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:17.523949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5ba1702-c7f5-460e-9580-77bd2d2a69fc None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:17.525339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5ba1702-c7f5-460e-9580-77bd2d2a69fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:17.563562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 869/1725] 10.222.0.31 () {58 vars in 1015 bytes} [Mon May 23 20:23:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:17.580990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:17.632286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:17.634343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:17.662943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1500c-0a8f-431d-9c6d-88f5d419cb2c service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:17.706233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:17.706993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:17.712444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:17.713463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:17.744351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:17.751171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7644ba-e5a3-47b7-b7c8-e83fd7f0827d service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:17.814836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 857/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:18.106894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:21.585816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16fed-09c8-4640-824b-19daa750ac78 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:21.586910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16fed-09c8-4640-824b-19daa750ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:21.587432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16fed-09c8-4640-824b-19daa750ac78 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:21.595986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 870/1727] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:23:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:21.622988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:21.623516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:21.623751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:21.690425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:21.710934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:21.886307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:21.996423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 858/1728] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:23:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:22.111450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771aef-4eb8-426f-af36-b339cb2d3f6e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:22.111695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771aef-4eb8-426f-af36-b339cb2d3f6e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:22.111956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771aef-4eb8-426f-af36-b339cb2d3f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:22.157581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a8771aef-4eb8-426f-af36-b339cb2d3f6e None None] MFA Rules not processed for user `954933c1cb0249c2a86b309b6f8a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:22.167661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a8771aef-4eb8-426f-af36-b339cb2d3f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:22.179362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:22.182315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:22.264064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df867b3a-3b58-467c-b211-0e71cd6fe8e2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:22.284574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:22.285366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:22.285717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:22.286144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:22.349557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:22.353877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 871/1729] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:22.362892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed8035-b189-45a6-9e38-074869d82ac1 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:22.451173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 859/1730] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:22.732001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:22.732498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:22.732582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:22.806155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] MFA Rules not processed for user `c4586fdc3e8b4ee0ab58d8c236637a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:22.807400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:22.844626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029c00-3d04-48e3-8e41-ef2c6d208af9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:22.845316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029c00-3d04-48e3-8e41-ef2c6d208af9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:22.845316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029c00-3d04-48e3-8e41-ef2c6d208af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:22.942774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cc029c00-3d04-48e3-8e41-ef2c6d208af9 None None] MFA Rules not processed for user `feb66e681a894c148e3463ff5a0dd1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:22.944542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cc029c00-3d04-48e3-8e41-ef2c6d208af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:23.031306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:23.107583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 872/1731] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:23.181019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:23.220293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff67f-8b80-4a53-a0ea-0f78bf498291 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:23.249582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:23.251630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 860/1732] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:23.254650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:23.254954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:23.256127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:23.258811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:23.260018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:23.260445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:23.327237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:23.329107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:23.356644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 873/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:23.360037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] MFA Rules not processed for user `feb66e681a894c148e3463ff5a0dd1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:23.361629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:23.394202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 861/1734] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:23.905864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:23.940231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7859-177d-43ab-83e9-6fb7b45d7b32 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:23.972549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:23.974888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:23.977826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:23.978334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:23.992171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:23.994294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1422b96-7e68-4273-88ef-59f3f48c94ae service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:24.036330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 874/1735] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:24.522324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:24.731597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e6595-9354-4e03-be54-2d3049157c4b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:24.740467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:24.741462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:24.741756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:24.742052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] PATH_INFO: `/v3/users/fc6ba6fb5a7a431c95484263d02efb63` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:24.743377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6ba6fb5a7a431c95484263d02efb63)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:24.746271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:24.939936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:24.947751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6ba6fb5a7a431c95484263d02efb63'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:24.948833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6ba6fb5a7a431c95484263d02efb63'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:24.950237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6ba6fb5a7a431c95484263d02efb63'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:24.966844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6ba6fb5a7a431c95484263d02efb63'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:24.977326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6ba6fb5a7a431c95484263d02efb63'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:25.064423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 862/1736] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:24 2022] DELETE /identity/v3/users/fc6ba6fb5a7a431c95484263d02efb63 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:25.069177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096d34-06d7-4b40-b69d-2dbc83614649 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:25.069779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096d34-06d7-4b40-b69d-2dbc83614649 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:25.069779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096d34-06d7-4b40-b69d-2dbc83614649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:25.103921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5096d34-06d7-4b40-b69d-2dbc83614649 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:25.105173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5096d34-06d7-4b40-b69d-2dbc83614649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:25.251884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 875/1737] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:25.414363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:25.416453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:25.604665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d425055-d72e-4b36-b462-9d106485209b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:25.616383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:25.617067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:25.617353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:25.617633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:25.651372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:25.653715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:25.678845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 863/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:27.358196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:27.359058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:27.359509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:27.383471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] MFA Rules not processed for user `feb66e681a894c148e3463ff5a0dd1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:27.384954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:27.679612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 876/1739] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:29.654910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:29.657130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:29.708846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7acc4-5101-4dde-bc57-ba545568f309 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:29.723961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:29.726171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:29.726171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:29.726171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] PATH_INFO: `/v3/projects/ee69177d6eed4ea390393c316e107099` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:29.736132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee69177d6eed4ea390393c316e107099)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:29.743208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:30.013411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee69177d6eed4ea390393c316e107099', 'request_id': 'req-9a61d04a-4ced-4b63-9a38-6376822129e5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.109031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee69177d6eed4ea390393c316e107099', 'request_id': 'req-9a61d04a-4ced-4b63-9a38-6376822129e5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.128970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee69177d6eed4ea390393c316e107099', 'request_id': 'req-9a61d04a-4ced-4b63-9a38-6376822129e5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.166682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] The token cache is being invalidate because project ee69177d6eed4ea390393c316e107099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:30.167022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.168463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9a61d04a-4ced-4b63-9a38-6376822129e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.174150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 864/1740] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:29 2022] DELETE /identity/v3/projects/ee69177d6eed4ea390393c316e107099 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:30.178997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:30.198914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc9425-a2c1-4648-824d-df53eb24817f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:30.511737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:30.513173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:30.513671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:30.514186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] PATH_INFO: `/v3/users/279daf7bcbdf4f7283f14ddf48129fcb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:30.520800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=279daf7bcbdf4f7283f14ddf48129fcb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:30.523423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:30.546085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:30.781142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279daf7bcbdf4f7283f14ddf48129fcb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.782619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279daf7bcbdf4f7283f14ddf48129fcb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.783712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279daf7bcbdf4f7283f14ddf48129fcb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.803649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279daf7bcbdf4f7283f14ddf48129fcb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.816271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f0a44ad6-0831-4b99-aa0c-3074fb745b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279daf7bcbdf4f7283f14ddf48129fcb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:30.943999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 877/1741] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:30 2022] DELETE /identity/v3/users/279daf7bcbdf4f7283f14ddf48129fcb => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:31.060768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:31.061560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:31.062081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:31.063962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 865/1742] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:23:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:31.091564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ff2-c57c-4fd6-b424-bee9ed3fff3b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:31.092604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ff2-c57c-4fd6-b424-bee9ed3fff3b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:31.093110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2ff2-c57c-4fd6-b424-bee9ed3fff3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:31.159945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b0ab2ff2-c57c-4fd6-b424-bee9ed3fff3b None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:31.166320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b0ab2ff2-c57c-4fd6-b424-bee9ed3fff3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:31.393062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 878/1743] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:31.563438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:31.565732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:31.638082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8366-1088-4580-854e-e6e1871f3e52 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:31.662923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:31.666663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:31.671431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:31.671911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:31.706708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:31.711585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:31.748323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 866/1744] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:31.884761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659de6b-d91e-4b7a-8209-165f8fe25f74 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:31.885228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659de6b-d91e-4b7a-8209-165f8fe25f74 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:31.885648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659de6b-d91e-4b7a-8209-165f8fe25f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:31.913674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6659de6b-d91e-4b7a-8209-165f8fe25f74 None None] MFA Rules not processed for user `c4586fdc3e8b4ee0ab58d8c236637a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:31.914766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6659de6b-d91e-4b7a-8209-165f8fe25f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:32.031084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 879/1745] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:32.195183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:32.196774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:32.259478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:32.341025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7159f2-88e0-43ba-80a9-5f280afaefa0 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:32.367067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:32.368427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:32.368876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:32.369302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] PATH_INFO: `/v3/projects/468f6516a2ef44c8be8edf1842e47907` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:32.371657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468f6516a2ef44c8be8edf1842e47907)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:32.375314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:32.554533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468f6516a2ef44c8be8edf1842e47907', 'request_id': 'req-af02d872-92b7-4479-9952-e89c9fbe67c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:32.676239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468f6516a2ef44c8be8edf1842e47907', 'request_id': 'req-af02d872-92b7-4479-9952-e89c9fbe67c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:32.687350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468f6516a2ef44c8be8edf1842e47907', 'request_id': 'req-af02d872-92b7-4479-9952-e89c9fbe67c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:32.719936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] The token cache is being invalidate because project 468f6516a2ef44c8be8edf1842e47907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:32.720276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:32.721066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-af02d872-92b7-4479-9952-e89c9fbe67c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:32.723999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 867/1746] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:32 2022] DELETE /identity/v3/projects/468f6516a2ef44c8be8edf1842e47907 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:32.730390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:32.730892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:32.731153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:32.774080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:32.779363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:32.805746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:32.865571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 880/1747] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:32.873933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:32.874128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:32.874330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:32.951837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:32.953315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:32.976622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:32.979602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 868/1748] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:32.992781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c71577-6576-47cd-9b9f-4261d7a481f1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.004325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.005019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.005389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.005742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.006436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:33.006743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.008439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.040831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 881/1749] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:33.042913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:33.061491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d8afa-2801-4e8f-a49e-780bb441afa0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.067646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.068618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.069057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.069525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.070508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.072550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.172393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 869/1750] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:33 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:33.176434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:33.200820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6d1a-c025-4251-9bdf-28fde7182ae0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.324454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.325666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.326037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.326542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.328188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.332799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.441722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 882/1751] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:33 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:33.446061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:33.457042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dcdbc-0161-4a88-a779-a3899f2b179b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.471034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.471034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.471034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.471034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.471369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.471369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.489617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 870/1752] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:33.493526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:33.512573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70455ff-213c-46cf-b29c-e3581315a824 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.525991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.528650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.529000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.529912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.530275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.532066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4291b9c-96c2-4111-a618-ffa9cc45028f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.551491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 883/1753] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:33.552889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:33.564709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f7c4e-68ab-4d27-8e26-ef43ed914a55 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:33.571585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:33.572491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.572886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.573303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] PATH_INFO: `/v3/projects/b2797cf6a2d34b29ac4715b727e79726/users/5524c02a69644ae285d5d1a9f133ccfb/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.589887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2797cf6a2d34b29ac4715b727e79726, user_id=5524c02a69644ae285d5d1a9f133ccfb, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:33.592870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:33.697847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 871/1754] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:33 2022] PUT /identity/v3/projects/b2797cf6a2d34b29ac4715b727e79726/users/5524c02a69644ae285d5d1a9f133ccfb/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:33.698679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:33.699426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:33.699939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:33.758495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] MFA Rules not processed for user `5524c02a69644ae285d5d1a9f133ccfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:33.761963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:33.871330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 884/1755] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:33.879448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.035162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ce773-313b-48ce-8238-be53d7daff88 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.042282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.043084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.043370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.043700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.044863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.046991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.167903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 872/1756] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:33 2022] POST /identity/v3/projects => generated 385 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:34.176010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.199731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db526b-4a13-4ae4-8b5a-a6601ba83dcf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.252758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:34.319723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.320410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.321292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.321292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.322602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.325738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.474354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 885/1757] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:34 2022] POST /identity/v3/users => generated 299 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:34.475881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.491417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d30807-b3df-43ef-aa19-de743d502908 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.504474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.505604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.506096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.506553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.507466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.509265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.531880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 873/1758] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:23:34.537658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.559983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433b96-12b3-4f00-8e52-b5a2ae16dfc3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.570489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.572890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.574805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.574996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] PATH_INFO: `/v3/projects/857af463216a49b8a6546478ec2ca9ee/users/7ab5bd6180f9491d887620ee9fdfcb66/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.613724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=857af463216a49b8a6546478ec2ca9ee, user_id=7ab5bd6180f9491d887620ee9fdfcb66, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.616707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.703527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 886/1759] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:34 2022] PUT /identity/v3/projects/857af463216a49b8a6546478ec2ca9ee/users/7ab5bd6180f9491d887620ee9fdfcb66/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:34.706650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.718910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51138a33-cfb3-4c50-9994-901d20b313f6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.807663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.809088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.809257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.809777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.810575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.812983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4abc-7cc8-4946-8a67-ecdb6b9d8255 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.832357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 874/1760] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:34.837018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:34.853965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23ed22-d94c-4bf3-988b-fdfd98aec068 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:34.864202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:34.882348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:34.883251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.884023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.884312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] PATH_INFO: `/v3/projects/857af463216a49b8a6546478ec2ca9ee/users/7ab5bd6180f9491d887620ee9fdfcb66/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:34.892719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] RBAC: Authorizing `identity:create_grant(project_id=857af463216a49b8a6546478ec2ca9ee, user_id=7ab5bd6180f9491d887620ee9fdfcb66, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:34.904359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:34.971269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 887/1761] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:34 2022] PUT /identity/v3/projects/857af463216a49b8a6546478ec2ca9ee/users/7ab5bd6180f9491d887620ee9fdfcb66/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:34.979415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:34.979709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:34.980045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:35.038539 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] MFA Rules not processed for user `7ab5bd6180f9491d887620ee9fdfcb66`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:35.042449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:35.168275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 875/1762] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:35.342900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:35.345310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:35.436018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b75b9d-805f-4196-a92f-480d950e987d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:35.439807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:35.440816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:35.440816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:35.441228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:35.461318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:35.463434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:35.485674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 888/1763] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:36.587524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:36.589352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:36.832431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:36.854705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5d57-e064-47a8-8b40-ed8fb230736e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:36.866290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:36.868732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:36.870360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:36.871235 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] PATH_INFO: `/v3/users/435ccc836ece4a4193cd0a89e7426667` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:36.873557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=435ccc836ece4a4193cd0a89e7426667)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:36.884335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:37.168242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435ccc836ece4a4193cd0a89e7426667'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.170529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435ccc836ece4a4193cd0a89e7426667'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.172403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435ccc836ece4a4193cd0a89e7426667'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.194911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435ccc836ece4a4193cd0a89e7426667'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.201663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a77bd523-797e-4080-9073-bd52150e4e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435ccc836ece4a4193cd0a89e7426667'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.280919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:37.294175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89108-9ebb-4df1-a0f0-db7e4f99e57f service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:37.301904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:37.308300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.308300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.308300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] PATH_INFO: `/v3/users/5524c02a69644ae285d5d1a9f133ccfb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.308611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 876/1764] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:36 2022] DELETE /identity/v3/users/435ccc836ece4a4193cd0a89e7426667 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:37.308699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5524c02a69644ae285d5d1a9f133ccfb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:37.309668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:37.311877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.312287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.312994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.358688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:37.360984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.495812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 877/1765] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:37.511589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:37.513703 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.540676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5524c02a69644ae285d5d1a9f133ccfb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.541488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5524c02a69644ae285d5d1a9f133ccfb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.542874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5524c02a69644ae285d5d1a9f133ccfb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.572206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5524c02a69644ae285d5d1a9f133ccfb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.584662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:37.587810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3cf639e0-fc3e-4b55-bb34-01f8c642c700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5524c02a69644ae285d5d1a9f133ccfb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:37.629941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737835-a530-472c-9fab-669ed4ef9050 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:37.638176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:37.638900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.639191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.639504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.644597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.672413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:37.673875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bcbd4-0913-44c1-927d-484d96c4443c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:37.693847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 878/1766] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:37.697795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d24e8-cb56-4200-9ff4-34229e00a55a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.697795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d24e8-cb56-4200-9ff4-34229e00a55a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.698190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d24e8-cb56-4200-9ff4-34229e00a55a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.702033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 879/1767] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:23:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:23:37.713114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.713976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.715013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.727659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 889/1768] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:37 2022] DELETE /identity/v3/users/5524c02a69644ae285d5d1a9f133ccfb => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:37.729084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.729376 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.729704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.737509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:37.738917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.773614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:37.774895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.865676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 880/1769] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:37.873735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:37.875851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.897686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 890/1770] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:37.904927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1eff4-d51a-4ea4-953a-eae16828f6da None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:37.911117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:37.911974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:37.912727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:37.913409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:37.922370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:37.922895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:37.924524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:37.938318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:37.941356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:37.965478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 881/1771] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:38.050325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd8c16-90dc-4048-b157-609c164f4f89 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:38.062237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:38.062990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:38.064844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:38.065197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:38.090670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:38.092626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32330f6a-1d16-4aa3-9ae8-fd6bebb7d055 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:38.117714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 891/1772] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:38.448554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:38.450575 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:38.465602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f99f4-a7de-43c3-bb8b-6a392884c6ef service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:38.471581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:38.472554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:38.473087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:38.473760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] PATH_INFO: `/v3/users/f62a3b686048478ea09c53c35f9c9921` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:38.474638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62a3b686048478ea09c53c35f9c9921)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:38.476905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:38.721924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62a3b686048478ea09c53c35f9c9921'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:38.722806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62a3b686048478ea09c53c35f9c9921'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:38.723584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62a3b686048478ea09c53c35f9c9921'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:38.741128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62a3b686048478ea09c53c35f9c9921'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:38.752372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62a3b686048478ea09c53c35f9c9921'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:38.846329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:38.847126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:38.847552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:38.853438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 882/1773] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:38 2022] DELETE /identity/v3/users/f62a3b686048478ea09c53c35f9c9921 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:38.867756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:38.869220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:38.956068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 892/1774] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:38.969667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:38.971197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.055039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.056784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.075918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135ef3c-c351-4195-a546-735dc4a4f407 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.077930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.083728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.084240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2f01b-65a8-427a-a17f-934fc7bee24b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.084540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.085004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.094165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.095142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.095653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.096085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] PATH_INFO: `/v3/projects/40888437d23e4f86958b7e7c5cdee27e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.097583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40888437d23e4f86958b7e7c5cdee27e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.100994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.118977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.120508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f01e16-eb75-454f-b5d6-a7e500f25d98 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.157533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 883/1775] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:39.224367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:39.233291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40888437d23e4f86958b7e7c5cdee27e', 'request_id': 'req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.300082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40888437d23e4f86958b7e7c5cdee27e', 'request_id': 'req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.306774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40888437d23e4f86958b7e7c5cdee27e', 'request_id': 'req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.317643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] The token cache is being invalidate because project 40888437d23e4f86958b7e7c5cdee27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:39.317935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.318615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.320158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 893/1776] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:39 2022] DELETE /identity/v3/projects/40888437d23e4f86958b7e7c5cdee27e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:39.325694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385a021-ff40-4a36-80af-743b440a2283 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.325982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385a021-ff40-4a36-80af-743b440a2283 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.326290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385a021-ff40-4a36-80af-743b440a2283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.344803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7385a021-ff40-4a36-80af-743b440a2283 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:39.345995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7385a021-ff40-4a36-80af-743b440a2283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.362409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.364410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.422459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 884/1777] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:39.428690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.428690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.428690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.448996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:39.450069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.458616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba01d-56f4-4e70-b2f5-303e8cbfa0a5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.467029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 885/1778] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:39.467211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.467211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.467211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.467211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] PATH_INFO: `/v3/projects/b2797cf6a2d34b29ac4715b727e79726` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.468632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.468748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2797cf6a2d34b29ac4715b727e79726)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.470752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.479269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c698ed-8ba1-4dae-be5a-72883c4b4322 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.484488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.485315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.485698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.486069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.487102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:39.487477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.489169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.505973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 886/1779] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:23:39.508468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.519478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981190-f0f4-4024-9a9a-350f5b1d4f9d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.524692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.525390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.526057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.526057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.527316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.529352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.546738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:39.578669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2797cf6a2d34b29ac4715b727e79726', 'request_id': 'req-8351ad2c-0fd7-4984-a938-56bcda3131aa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.624507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 887/1780] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:39 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:39.626140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.629753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.660955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2797cf6a2d34b29ac4715b727e79726', 'request_id': 'req-8351ad2c-0fd7-4984-a938-56bcda3131aa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.668308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2797cf6a2d34b29ac4715b727e79726', 'request_id': 'req-8351ad2c-0fd7-4984-a938-56bcda3131aa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.687772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] The token cache is being invalidate because project b2797cf6a2d34b29ac4715b727e79726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:39.687772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.688881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.690104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 894/1781] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:39 2022] DELETE /identity/v3/projects/b2797cf6a2d34b29ac4715b727e79726 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:39.693091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.694820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.722189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc654c2b-8800-4c2d-a76a-8c828e0dda01 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.728363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.729265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.729583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.729847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] PATH_INFO: `/v3/projects/5642c4590d4c440fb835a76ba9f0869a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.731176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5642c4590d4c440fb835a76ba9f0869a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.733435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.782946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351ad2c-0fd7-4984-a938-56bcda3131aa admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.788378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:39.789051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:39.789391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:39.789670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:39.790404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:39.792359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:39.864872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5642c4590d4c440fb835a76ba9f0869a', 'request_id': 'req-9765192b-d3f9-41b3-971f-0184ed37dab3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.894953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 895/1782] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:39 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:39.897429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.897860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.911224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c4c57-f6b5-4bb4-934e-8a2cfb7933a6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:39.945319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5642c4590d4c440fb835a76ba9f0869a', 'request_id': 'req-9765192b-d3f9-41b3-971f-0184ed37dab3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.952577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5642c4590d4c440fb835a76ba9f0869a', 'request_id': 'req-9765192b-d3f9-41b3-971f-0184ed37dab3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.969952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] The token cache is being invalidate because project 5642c4590d4c440fb835a76ba9f0869a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:39.970380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.971148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:39.976515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 888/1783] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:39 2022] DELETE /identity/v3/projects/5642c4590d4c440fb835a76ba9f0869a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:39.977990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:39.979410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:39.999230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:40.000894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.001362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.001823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] PATH_INFO: `/v3/users/7ab5bd6180f9491d887620ee9fdfcb66` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.004431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab5bd6180f9491d887620ee9fdfcb66)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:40.007236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:40.070289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765192b-d3f9-41b3-971f-0184ed37dab3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:40.075628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:40.076549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.077096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.077846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.078818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:40.080922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:40.093395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 889/1784] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:40.095111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:40.097079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:40.111613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874819a-8683-4188-bd07-cbd86e8fd16b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:40.116438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:40.117296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.117550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.117826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] PATH_INFO: `/v3/users/2d01f1c4113a495a90b1e627eb48b1cb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.120155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d01f1c4113a495a90b1e627eb48b1cb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:40.121915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:40.243837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab5bd6180f9491d887620ee9fdfcb66'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.244528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab5bd6180f9491d887620ee9fdfcb66'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.245166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab5bd6180f9491d887620ee9fdfcb66'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.260859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab5bd6180f9491d887620ee9fdfcb66'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.270099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab5bd6180f9491d887620ee9fdfcb66'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.397834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d01f1c4113a495a90b1e627eb48b1cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.400775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d01f1c4113a495a90b1e627eb48b1cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.403005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d01f1c4113a495a90b1e627eb48b1cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.421512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 896/1785] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:39 2022] DELETE /identity/v3/users/7ab5bd6180f9491d887620ee9fdfcb66 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:40.426868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:40.431420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:40.431877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d01f1c4113a495a90b1e627eb48b1cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.449374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d01f1c4113a495a90b1e627eb48b1cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:40.568352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 890/1786] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:40 2022] DELETE /identity/v3/users/2d01f1c4113a495a90b1e627eb48b1cb => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:40.587209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:40.594736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5d6ed-f365-4219-99bb-b1970a5d640f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:40.684989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:40.686501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.686892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.687417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.688939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:40.690966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c8185-a3bb-48e9-baed-8ce746c11274 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:40.709969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 897/1787] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:40.711466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:40.712913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:40.727749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7002-a0c4-4ad4-9751-1d726f8823d8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:40.734367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:40.735331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.736203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.736659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] PATH_INFO: `/v3/projects/620af9bcc4e74057817da667d4dddf22/users/6f658d9f50f347938e8f66ededa27380/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.767405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620af9bcc4e74057817da667d4dddf22, user_id=6f658d9f50f347938e8f66ededa27380, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:40.770174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:40.902357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 891/1788] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:40 2022] PUT /identity/v3/projects/620af9bcc4e74057817da667d4dddf22/users/6f658d9f50f347938e8f66ededa27380/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:40.913471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:40.914041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:40.914041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:40.961862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] MFA Rules not processed for user `6f658d9f50f347938e8f66ededa27380`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:40.963749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:41.013315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:41.056223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 898/1789] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:41.074430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:41.077043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:41.214800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab99-984c-447b-a671-34a8735d1fd1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:41.218274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:41.219789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:41.220913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:41.221277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:41.233219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:41.234608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:41.248031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 892/1790] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:41.274959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:41.396721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba68a-8f7e-428e-b5c5-b24bcdec468b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:41.405730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:41.406657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:41.406935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:41.407760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] PATH_INFO: `/v3/users/6f658d9f50f347938e8f66ededa27380` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:41.411008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f658d9f50f347938e8f66ededa27380)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:41.415718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:41.652852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f658d9f50f347938e8f66ededa27380'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.653527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f658d9f50f347938e8f66ededa27380'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.658847 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f658d9f50f347938e8f66ededa27380'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.668632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f658d9f50f347938e8f66ededa27380'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.677435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f658d9f50f347938e8f66ededa27380'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.713255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:41.714787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:41.731276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f44a-30f3-4eea-8419-03f448045f1e service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:41.736319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:41.737213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:41.737565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:41.737832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] PATH_INFO: `/v3/projects/32312c5c1cc34f2199d8b84003f6a112` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:41.739115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32312c5c1cc34f2199d8b84003f6a112)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:41.741063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:41.815017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 899/1791] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:41 2022] DELETE /identity/v3/users/6f658d9f50f347938e8f66ededa27380 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:41.815894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:41.817254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:41.898371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32312c5c1cc34f2199d8b84003f6a112', 'request_id': 'req-62f592d7-46ea-48e4-b9b8-328f2fc96822'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.906479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73374-1ba4-4bb4-bfe9-736b13ea72d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:41.990157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32312c5c1cc34f2199d8b84003f6a112', 'request_id': 'req-62f592d7-46ea-48e4-b9b8-328f2fc96822'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:41.997705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32312c5c1cc34f2199d8b84003f6a112', 'request_id': 'req-62f592d7-46ea-48e4-b9b8-328f2fc96822'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.005184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.006103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.006550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.006971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] PATH_INFO: `/v3/projects/857af463216a49b8a6546478ec2ca9ee` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.008411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857af463216a49b8a6546478ec2ca9ee)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.011147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.029996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] The token cache is being invalidate because project 32312c5c1cc34f2199d8b84003f6a112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:42.031536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.036776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-62f592d7-46ea-48e4-b9b8-328f2fc96822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.038654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 893/1792] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:41 2022] DELETE /identity/v3/projects/32312c5c1cc34f2199d8b84003f6a112 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:42.040796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe464-cf24-4489-87c2-ed08444d3525 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.046171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe464-cf24-4489-87c2-ed08444d3525 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.046874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe464-cf24-4489-87c2-ed08444d3525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.074263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f0bfe464-cf24-4489-87c2-ed08444d3525 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:42.076541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f0bfe464-cf24-4489-87c2-ed08444d3525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.088049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 894/1793] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:42.090232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.090992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.091193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.111365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:42.112702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.122761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 895/1794] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:42.123863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.125471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.131259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857af463216a49b8a6546478ec2ca9ee', 'request_id': 'req-b26d146e-ca83-425f-897e-943d3faa0e48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.192711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076fac-ec16-4b2d-8210-16dbec4e96d1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.194933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857af463216a49b8a6546478ec2ca9ee', 'request_id': 'req-b26d146e-ca83-425f-897e-943d3faa0e48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.203563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857af463216a49b8a6546478ec2ca9ee', 'request_id': 'req-b26d146e-ca83-425f-897e-943d3faa0e48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.217444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] The token cache is being invalidate because project 857af463216a49b8a6546478ec2ca9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:42.217736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.218858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b26d146e-ca83-425f-897e-943d3faa0e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:42.220661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 900/1795] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:41 2022] DELETE /identity/v3/projects/857af463216a49b8a6546478ec2ca9ee => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:42.221708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6664c7-edf5-49d9-875f-67c17e7de8f7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.221997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6664c7-edf5-49d9-875f-67c17e7de8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.222301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6664c7-edf5-49d9-875f-67c17e7de8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.249222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bd6664c7-edf5-49d9-875f-67c17e7de8f7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:42.250583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bd6664c7-edf5-49d9-875f-67c17e7de8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.260939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.261599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.261885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.262198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.266363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.341587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 901/1796] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:42.343004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cc94-8f77-49ba-922d-90655a145006 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.343345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cc94-8f77-49ba-922d-90655a145006 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.343632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cc94-8f77-49ba-922d-90655a145006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.375664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.377607 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.387466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:42.393709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 896/1797] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:42.394904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.403526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9499cc94-8f77-49ba-922d-90655a145006 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:42.406974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9499cc94-8f77-49ba-922d-90655a145006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.408057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddf1cf-b4e8-4c6e-bea7-f6fe0c3bdd3c service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.415213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.416586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.416944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.417669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.419254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:42.419875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.422723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.428293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 902/1798] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:42.436454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.436740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.437017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.459896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 897/1799] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:42.461265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.485022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:42.488790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:42.489927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0d1f-769a-42fd-b98c-5e9cfd122ac6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.501305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.504094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.504669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.504933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.505651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.513414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.513589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 903/1800] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:42.514507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.535585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7f289-41f4-4b2f-964f-ef8088ed8c0c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.553056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.553892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.554293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.564424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.564424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:42.564424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.564424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.572720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 904/1801] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:42.577875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.596210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e8ab-7d24-4ac6-bac6-98c18c009936 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.602617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.603448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.603966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.604335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.606148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.608996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.618454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:42.660277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 898/1802] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:42.664241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.751604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 905/1803] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:42.756239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:42.812426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469aa-1cf8-4e57-91c4-b9a8b180e937 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.929343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.930064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.930350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.930623 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.931683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.934876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca48fb-4032-4ec6-90af-863626054a15 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:42.936380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:42.951832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:42.952467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:42.953059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:42.953175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:42.953872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:42.956160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.074473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 906/1804] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:43.082759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 899/1805] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:42 2022] POST /identity/v3/users => generated 294 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:23:43.083829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.085137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.105826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d7e2-a815-4313-accf-efc8723055d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.107213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600fbe-98e0-4506-b0d0-2c3119a7aa53 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.114320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.115026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.115316 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.115599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.115699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.116344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.116629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.117661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.117962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.118082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.119411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.121177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.129047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 907/1806] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:43.129981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.132144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.135033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 900/1807] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:43.137011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.149197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b70ae-a85e-4d69-8607-c53266b63a98 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.149341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209fc9-5ba1-4f78-9469-8358689b1510 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.157060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.158446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.158446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.158648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] PATH_INFO: `/v3/projects/620af9bcc4e74057817da667d4dddf22` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.159775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620af9bcc4e74057817da667d4dddf22)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.162054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.165414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.167409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.167409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.169943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.171363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.176510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.197971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 901/1808] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:43.199617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.211724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae48e-429c-43a4-b1ff-2447d6d03c56 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.222601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.223293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.223696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.226396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.227172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.230283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.245631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 902/1809] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:43.245631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.254840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ffb15-aaad-4882-bbb7-918df9e5104f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.260440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.262453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.262735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.263012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] PATH_INFO: `/v3/projects/2253decf5fa94603babbf5c9e8c0139f/users/f575798d78294565b37bb2daf458dd21/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.284881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620af9bcc4e74057817da667d4dddf22', 'request_id': 'req-fe9208f2-28b9-43b3-adc0-a3904a0191e2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:43.291750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2253decf5fa94603babbf5c9e8c0139f, user_id=f575798d78294565b37bb2daf458dd21, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.294618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.372412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620af9bcc4e74057817da667d4dddf22', 'request_id': 'req-fe9208f2-28b9-43b3-adc0-a3904a0191e2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:43.380191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620af9bcc4e74057817da667d4dddf22', 'request_id': 'req-fe9208f2-28b9-43b3-adc0-a3904a0191e2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:43.392408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 903/1810] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:43 2022] PUT /identity/v3/projects/2253decf5fa94603babbf5c9e8c0139f/users/f575798d78294565b37bb2daf458dd21/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:43.392882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] The token cache is being invalidate because project 620af9bcc4e74057817da667d4dddf22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:43.393336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:43.393908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.394148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fe9208f2-28b9-43b3-adc0-a3904a0191e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:43.396212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 908/1811] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:43 2022] DELETE /identity/v3/projects/620af9bcc4e74057817da667d4dddf22 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:43.400865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182ec9-c7ae-468f-b275-d3f7da3c4d85 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.401056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182ec9-c7ae-468f-b275-d3f7da3c4d85 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.401335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182ec9-c7ae-468f-b275-d3f7da3c4d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.403458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.419872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-04182ec9-c7ae-468f-b275-d3f7da3c4d85 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:43.421145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-04182ec9-c7ae-468f-b275-d3f7da3c4d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.487739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b117b83-1366-4d2f-a257-82df1e1e7527 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.493480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.494893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.495876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.497503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 909/1812] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:43.499442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4aaa-62ec-459f-ab51-a2c415768d33 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.499863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4aaa-62ec-459f-ab51-a2c415768d33 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.500230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4aaa-62ec-459f-ab51-a2c415768d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.501925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] PATH_INFO: `/v3/projects/78c929a145cb4695b54c059b547e0424/users/7971f95fcaae4698871f8b698fdc9479/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.532411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f00b4aaa-62ec-459f-ab51-a2c415768d33 None None] MFA Rules not processed for user `f575798d78294565b37bb2daf458dd21`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:43.533907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f00b4aaa-62ec-459f-ab51-a2c415768d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.548236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c929a145cb4695b54c059b547e0424, user_id=7971f95fcaae4698871f8b698fdc9479, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.552378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.612935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 910/1813] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:43.614357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.614738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.615116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.635730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:43.637631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.647293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 904/1814] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:43 2022] PUT /identity/v3/projects/78c929a145cb4695b54c059b547e0424/users/7971f95fcaae4698871f8b698fdc9479/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:43.647665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 911/1815] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:43.648437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.652116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.652249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.662197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465b04a-6275-4559-9cda-d689340ccf08 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.730621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.731455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.731827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.732173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.732506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e527-37ea-4699-8f39-dae5df7a6747 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.733078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:43.733385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.735017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12418078-557e-4649-bda9-6d95698a3161 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.737494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.738275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.738631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.739002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.739829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.741803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.753005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 912/1816] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:43.753005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.753005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.753005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.778432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] MFA Rules not processed for user `7971f95fcaae4698871f8b698fdc9479`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:43.779650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:43.841391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 905/1817] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:43.842833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.849868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 913/1818] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:43.850800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:43.866504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a988d3-3f59-425d-a14e-f241ab638ba7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.921079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417606a-1c89-4e07-b2a0-f68c308e6b5a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:43.937697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.938416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.938699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.939063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.939772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.942814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:43.943366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:43.943681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:43.944041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:43.944390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:43.945161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:43.947295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.084368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 914/1819] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:44.085204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.090452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 906/1820] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:43 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:44.091174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.103367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d423a-8d55-45ba-a44a-ea51cef5dbb4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.180622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8114b-1faa-4417-9dd2-32c7bc31c7ce admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.185811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.186588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.186978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.187346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.188311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.188532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.190430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.190430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.190430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.190719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.191054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.193085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.204605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 907/1821] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:44.206097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.226443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16161d0f-d6c6-417b-884d-f729cfe88162 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.235880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.236748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.237222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.237633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.238510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.240664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.306887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 915/1822] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:44 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:44.307975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.325920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 908/1823] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:44 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:44.325920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.345507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26697-1586-4268-a8ce-3a7fcad536b5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.408568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a8f8-44c3-48c7-aa6a-e19f51483e99 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.418176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.419200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.419558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.419907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] PATH_INFO: `/v3/projects/856246beca554beb923fa3adfa77aa1e/users/5d1e41bf665647af878e8f9191efef02/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.450061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856246beca554beb923fa3adfa77aa1e, user_id=5d1e41bf665647af878e8f9191efef02, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.458846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.469823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.470806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.471280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.472250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.473301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.475877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.569007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 916/1824] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:44 2022] PUT /identity/v3/projects/856246beca554beb923fa3adfa77aa1e/users/5d1e41bf665647af878e8f9191efef02/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:44.572687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.600534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 909/1825] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:44 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:44.601456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.629835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff441-d0d5-4093-9351-e86ecf97c1d0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.684096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:44.716533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449f9e7-21be-44ff-b38a-08d8cf475246 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.729428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.729683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.730191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.732214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.732444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.732551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.732627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.732943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.733023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.733091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.733547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.734980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.757321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 910/1826] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:44.758878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.764945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 917/1827] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:44.768769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.772284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430cb4a-8cf9-408a-a2e8-9dd6900bd6a0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.779179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.780195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.780731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.781237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.782240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.785596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.788720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f578230-ffd2-4e50-917e-b12be07aa73a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.804120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 911/1828] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:44.805477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.809028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.809956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.810268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.810971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] PATH_INFO: `/v3/projects/856246beca554beb923fa3adfa77aa1e/users/5d1e41bf665647af878e8f9191efef02/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.812709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856246beca554beb923fa3adfa77aa1e, user_id=5d1e41bf665647af878e8f9191efef02, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.819934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.826453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995ebc2-1cdc-49d6-8380-745bf187a234 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.835728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 918/1829] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:44 2022] PUT /identity/v3/projects/856246beca554beb923fa3adfa77aa1e/users/5d1e41bf665647af878e8f9191efef02/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:44.836618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:44.924591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144f3-765a-4b89-8f12-7c44533269b4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:44.932131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.933794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.934344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.934786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] PATH_INFO: `/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2/users/0ce640e66bc0430594a44dd702d81f2f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.935884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:44.936755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.937141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.937520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.938583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.940422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49470135-362d-4302-ad00-a9ce1261009a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.953990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 912/1830] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:44.955545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:44.955837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:44.956761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:44.965122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2328a1b5ff471abbeab6db4746bdb2, user_id=0ce640e66bc0430594a44dd702d81f2f, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:44.968609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:44.991043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] MFA Rules not processed for user `5d1e41bf665647af878e8f9191efef02`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:44.992397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:45.049669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 919/1831] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:44 2022] PUT /identity/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2/users/0ce640e66bc0430594a44dd702d81f2f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:45.052622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.126972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 913/1832] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:45.127896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.149929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6c9f-6f75-44d9-bc73-2c5cc59fea5e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.155763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:45.186840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b0dcc-914e-4a1d-8de5-1d04569d5a80 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.192978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.194542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.194937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.195974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] PATH_INFO: `/v3/projects/da02174b2ced4bc29ab2688199043787/users/20146cad93ce44b8b4a483726e14ad3a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.216269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da02174b2ced4bc29ab2688199043787, user_id=20146cad93ce44b8b4a483726e14ad3a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.219391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.256476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.257294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.257579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.257854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.258632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.260237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.273194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 914/1833] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:45.274187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.277788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:45.311524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 920/1834] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:45 2022] PUT /identity/v3/projects/da02174b2ced4bc29ab2688199043787/users/20146cad93ce44b8b4a483726e14ad3a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:45.311524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.349767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de57b21-7518-4926-9418-82277594d0bc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.393580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512eaa5-0cdb-4a0a-86c6-9fb037deb2eb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.399263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.400785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.401277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.401932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] PATH_INFO: `/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2/users/0ce640e66bc0430594a44dd702d81f2f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.405623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2328a1b5ff471abbeab6db4746bdb2, user_id=0ce640e66bc0430594a44dd702d81f2f, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.409524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf68cf-025b-4326-85f8-b55a43534911 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.413013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.413842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.414212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.414553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.426124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 921/1835] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:45 2022] PUT /identity/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2/users/0ce640e66bc0430594a44dd702d81f2f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:45.427644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.428305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.428655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.466804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] MFA Rules not processed for user `20146cad93ce44b8b4a483726e14ad3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:45.468217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:45.502633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.504322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de64d90-c121-4f4f-b88a-84ed528f0f26 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.514275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 915/1836] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:45.515443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.515737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.516015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.550927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 922/1837] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:45.555129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.557581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] MFA Rules not processed for user `0ce640e66bc0430594a44dd702d81f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:45.558889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:45.677398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d942c-6e9b-4a2e-a68e-f1579852651f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.691188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.691944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.692232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.692532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.693229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.696380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.703297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 916/1838] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:45.800263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 923/1839] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:45 2022] POST /identity/v3/projects => generated 375 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:23:45.804309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.819023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643cde0-26f9-4def-ac22-166a50ae26f7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.879520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:45.881291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:45.896909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.897867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.898347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.898982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:45.899661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:45.902483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:45.959181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c723f-ac4a-4c5e-807a-09e230e25fd1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:45.964471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:45.965291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:45.965576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:45.965854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.010412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 917/1840] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:45 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:23:46.012281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.034438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb86d4-149a-4875-84fa-cb86fbf76ed4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.043293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.043996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.045025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.045312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.046108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.048746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.065707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 918/1841] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:46.070762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.091645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e679d4d-f602-4bd4-8c08-672ba1664ffb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.092164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.093931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.101386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.102901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.103343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.106231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] PATH_INFO: `/v3/projects/75466547f0554439adf0a20b38e63d3d/users/1d1f987bc40c4ac0ae9104ea0f897829/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.117439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 924/1842] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:46.124856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75466547f0554439adf0a20b38e63d3d, user_id=1d1f987bc40c4ac0ae9104ea0f897829, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.129003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.154817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.165574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbdf1-e945-4258-a690-bbc0c42670f6 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.170691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.171810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.172202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.172744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] PATH_INFO: `/v3/users/7971f95fcaae4698871f8b698fdc9479` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.173965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7971f95fcaae4698871f8b698fdc9479)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.176027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.256780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 919/1843] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:46 2022] PUT /identity/v3/projects/75466547f0554439adf0a20b38e63d3d/users/1d1f987bc40c4ac0ae9104ea0f897829/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:46.257745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.368453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b2b8-75f1-4b3c-aae7-6adb0814af6a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.375520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7971f95fcaae4698871f8b698fdc9479'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:46.376012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.376587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7971f95fcaae4698871f8b698fdc9479'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:46.376757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.377157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.377275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7971f95fcaae4698871f8b698fdc9479'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:46.377517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.378526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.380409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.391242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7971f95fcaae4698871f8b698fdc9479'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:46.396489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 920/1844] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:46.398166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.399117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-295224d0-c6ad-471b-a031-76efcd92ef11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7971f95fcaae4698871f8b698fdc9479'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:46.399823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.415569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92237ff0-42aa-4bd6-8172-ac75629932d0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.420709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.421881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.421881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.422204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] PATH_INFO: `/v3/projects/75466547f0554439adf0a20b38e63d3d/users/1d1f987bc40c4ac0ae9104ea0f897829/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.424136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75466547f0554439adf0a20b38e63d3d, user_id=1d1f987bc40c4ac0ae9104ea0f897829, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.426829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392619d0-df78-4476-a8a8-9e2fbdc9e802 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.442100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 921/1845] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:46 2022] PUT /identity/v3/projects/75466547f0554439adf0a20b38e63d3d/users/1d1f987bc40c4ac0ae9104ea0f897829/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:46.449975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.450298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.450580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.480415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] MFA Rules not processed for user `1d1f987bc40c4ac0ae9104ea0f897829`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:46.481826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.487384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 925/1846] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:46 2022] DELETE /identity/v3/users/7971f95fcaae4698871f8b698fdc9479 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:46.488834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.489131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.489415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.515715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:46.516950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.571789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 922/1847] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:46.586144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.588499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.597824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 926/1848] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:46.604345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:46.606129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.669623 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44445def-18ee-45c1-9a36-3b563b123b87 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.674790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.675632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.676120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.676584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.679254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:46.686095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623539f0-12a1-4a88-b5ac-99c24c7a0945 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:46.691631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:46.692290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:46.692904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:46.693455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:46.703758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.705671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.716577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 927/1849] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:46.750396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:46.751958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:46.761288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 923/1850] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:47.030893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:47.273008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:47.274556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:47.313640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcaac0-62f6-4f5b-af77-a7a697a6271a service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:47.318531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:47.320545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:47.320545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:47.320545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] PATH_INFO: `/v3/projects/78c929a145cb4695b54c059b547e0424` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:47.321663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c929a145cb4695b54c059b547e0424)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:47.324314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:47.478524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c929a145cb4695b54c059b547e0424', 'request_id': 'req-88598485-2974-48b6-800b-c8a5fb09af3c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.580341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c929a145cb4695b54c059b547e0424', 'request_id': 'req-88598485-2974-48b6-800b-c8a5fb09af3c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.587501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c929a145cb4695b54c059b547e0424', 'request_id': 'req-88598485-2974-48b6-800b-c8a5fb09af3c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.598711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] The token cache is being invalidate because project 78c929a145cb4695b54c059b547e0424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:47.599077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.599824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.601372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 928/1851] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:47 2022] DELETE /identity/v3/projects/78c929a145cb4695b54c059b547e0424 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:47.605881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:47.616699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b820c0-02a7-4ba6-b38a-627ba7212cbd service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:47.690359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:47.704300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:47.704300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:47.704453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] PATH_INFO: `/v3/users/0ce640e66bc0430594a44dd702d81f2f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:47.707680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ce640e66bc0430594a44dd702d81f2f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:47.709918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:47.953644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ce640e66bc0430594a44dd702d81f2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.954437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ce640e66bc0430594a44dd702d81f2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.955298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ce640e66bc0430594a44dd702d81f2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.967733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ce640e66bc0430594a44dd702d81f2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:47.974410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1a659ea-a1a8-437c-b1b6-9d9b55d0a5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ce640e66bc0430594a44dd702d81f2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:48.071603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 924/1852] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:47 2022] DELETE /identity/v3/users/0ce640e66bc0430594a44dd702d81f2f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:48.789612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:48.791305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:48.804912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88598485-2974-48b6-800b-c8a5fb09af3c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:48.874594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:48.875817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:48.876405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:48.876996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] PATH_INFO: `/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:48.879390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2328a1b5ff471abbeab6db4746bdb2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:48.882116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.045705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2328a1b5ff471abbeab6db4746bdb2', 'request_id': 'req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:49.089410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:49.173711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2328a1b5ff471abbeab6db4746bdb2', 'request_id': 'req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:49.180870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2328a1b5ff471abbeab6db4746bdb2', 'request_id': 'req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:49.192743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] The token cache is being invalidate because project 2a2328a1b5ff471abbeab6db4746bdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:49.193097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:49.193816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2e4c3210-0a7f-4c72-9dcf-1dea0c86403c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:49.195632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 929/1853] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:48 2022] DELETE /identity/v3/projects/2a2328a1b5ff471abbeab6db4746bdb2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:49.201526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.202456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.202886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.223544 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:49.225086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:49.314216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 925/1854] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:49.317525 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.317816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.318131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.340948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:49.342209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:49.356381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 930/1855] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:49.360203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.374701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5e3b2-3881-4658-b2b5-6a56b80a8072 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.379400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.380086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.380345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.390739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.390739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:49.390739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.390739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.395781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 926/1856] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:23:49.400223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.410399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72609c7e-7c0d-4742-a251-46eb97792469 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.415255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.415926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.416214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.416493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.417761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.419713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.492590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 931/1857] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:49 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:49.493859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.504689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832766-7b84-44a1-8e3b-b279c2a828e3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.576332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.577181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.577533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.578287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.581515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.585132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.725918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 927/1858] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:49 2022] POST /identity/v3/users => generated 284 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:49.730145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.742305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754691c5-2ede-4cc0-ae8e-76c89c57ae1f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.750288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.755305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.756174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.756830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.758504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.764752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.777667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 932/1859] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:49.779412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.793755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc82cf-fa3e-4005-b94d-7e9ae209a6de admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.801108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.801580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.801882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.802188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.802986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.804665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.826245 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 928/1860] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:49.827396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.841380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c06b-2655-4394-ac47-8474a73dbb44 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.847635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.848916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.849448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.849932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] PATH_INFO: `/v3/projects/9f66c498128c4a66832a961694744ec3/users/8e69faf2109547bf8980b57e8f5bcc44/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.870671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f66c498128c4a66832a961694744ec3, user_id=8e69faf2109547bf8980b57e8f5bcc44, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.875074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38ad8-1d3c-4498-a75b-d9692c77a6dc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.953241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:49.955559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:49.969316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d91b7-ff92-4585-afb9-ee6c01c4dd51 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:49.974103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:49.975280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.975416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.975602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] PATH_INFO: `/v3/users/20146cad93ce44b8b4a483726e14ad3a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:49.977946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20146cad93ce44b8b4a483726e14ad3a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:49.981172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:49.993341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 933/1861] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:49 2022] PUT /identity/v3/projects/9f66c498128c4a66832a961694744ec3/users/8e69faf2109547bf8980b57e8f5bcc44/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:49.995970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:49.996268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:49.996551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.059559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] MFA Rules not processed for user `8e69faf2109547bf8980b57e8f5bcc44`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:50.060919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:50.143831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 934/1862] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:50.147855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.208875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20146cad93ce44b8b4a483726e14ad3a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:50.210105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20146cad93ce44b8b4a483726e14ad3a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:50.215078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20146cad93ce44b8b4a483726e14ad3a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:50.229929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ad19-2321-49bb-9e92-4ab3c4bd0713 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.234784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20146cad93ce44b8b4a483726e14ad3a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:50.236989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.237736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.238251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.238593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.239360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.241484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.241894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-519f3dd6-e428-434e-bdf6-3873e90b095e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20146cad93ce44b8b4a483726e14ad3a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:50.344510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 929/1863] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:49 2022] DELETE /identity/v3/users/20146cad93ce44b8b4a483726e14ad3a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:50.349506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 935/1864] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:50 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:23:50.350721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.351141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.351410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.352128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.353719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:50.376925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:50.378841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:50.429791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9c9a-2772-4fc6-b3c3-87a38b0f24cf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.436747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.437480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.437710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.437986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.438729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.440987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.449811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 930/1865] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:50.461120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.462588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:50.518903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963323dd-b1e5-4a68-b129-c9fbe12e5d83 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.522696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.523347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.523831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.524106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.530226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 936/1866] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:50 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:50.532992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.537956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.539426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.543713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe006e6-4509-4641-abd2-f3087f9f26f9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.549918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 931/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:50.556098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.556827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.557117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.557449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.572170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.572170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.572655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 937/1868] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:50.577080 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.586412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88120a14-2c6d-48c3-b1a2-b60979a2149d service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.591785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.593388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.593388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.593388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] PATH_INFO: `/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8/users/50412b7b06f14cdd9394d43a4324d8ea/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.609922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911d4fe9c4364fe89ea1ca97b9a267e8, user_id=50412b7b06f14cdd9394d43a4324d8ea, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.612803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:50.613830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.737195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 932/1869] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:50 2022] PUT /identity/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8/users/50412b7b06f14cdd9394d43a4324d8ea/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:50.740961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.751700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df5b4a-401a-43ab-8a68-d9fb61449ccc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.823657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.824350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.824640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.824915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.825653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.827304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127ebc-808c-4a79-86b3-a5c7e815ca65 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.842623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 938/1870] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:50.844158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.858432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7696c-87ed-495f-9e85-61e11c99ec86 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:50.865510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:50.865510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.865510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.865510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] PATH_INFO: `/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8/users/50412b7b06f14cdd9394d43a4324d8ea/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.867311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=911d4fe9c4364fe89ea1ca97b9a267e8, user_id=50412b7b06f14cdd9394d43a4324d8ea, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:50.870193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:50.883176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 933/1871] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:50 2022] PUT /identity/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8/users/50412b7b06f14cdd9394d43a4324d8ea/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:50.893719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:50.894092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:50.894306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:50.926043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] MFA Rules not processed for user `50412b7b06f14cdd9394d43a4324d8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:50.927409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:50.977734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:50.982562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.015164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 939/1872] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:51.027907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:51.029340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.044112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:51.083678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea002e8-7714-49b9-a9fa-1a4d18c8b28e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:51.097220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:51.098780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:51.100088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:51.100608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] PATH_INFO: `/v3/projects/da02174b2ced4bc29ab2688199043787` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:51.103443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da02174b2ced4bc29ab2688199043787)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:51.106791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:51.163657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732796d-64d3-4456-a3e6-fec396ed13b8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:51.175219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:51.181173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:51.181173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:51.181173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:51.205193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:51.209925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:51.227932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 940/1873] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:51.270292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da02174b2ced4bc29ab2688199043787', 'request_id': 'req-c30df35a-0afc-4fe7-8849-8230bfd96a1f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.359433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:51.361013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.372610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b243c-c4cf-4d83-9ed3-23e236e89e84 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:51.389397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da02174b2ced4bc29ab2688199043787', 'request_id': 'req-c30df35a-0afc-4fe7-8849-8230bfd96a1f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.392885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da02174b2ced4bc29ab2688199043787', 'request_id': 'req-c30df35a-0afc-4fe7-8849-8230bfd96a1f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.413437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] The token cache is being invalidate because project da02174b2ced4bc29ab2688199043787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:51.413734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.415791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.417952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 934/1874] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:50 2022] DELETE /identity/v3/projects/da02174b2ced4bc29ab2688199043787 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:51.423795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:51.425205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.479036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:51.479759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:51.480050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:51.480325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:51.482765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.550136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df35a-0afc-4fe7-8849-8230bfd96a1f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:51.570163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:51.571286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:51.571578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:51.571854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] PATH_INFO: `/v3/users/1d1f987bc40c4ac0ae9104ea0f897829` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:51.580957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1f987bc40c4ac0ae9104ea0f897829)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:51.583437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:51.638262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:51.639837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:51.668573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 941/1875] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:51.669932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:51.674195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.704860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3025322-47fb-4839-b750-71c233c08197 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:51.709144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:51.709812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:51.710123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:51.710405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:51.714646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:51.817615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1f987bc40c4ac0ae9104ea0f897829'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.819015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1f987bc40c4ac0ae9104ea0f897829'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.821091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1f987bc40c4ac0ae9104ea0f897829'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.857152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1f987bc40c4ac0ae9104ea0f897829'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.866070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:51.869880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:51.874133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ce4731df-8c66-40da-af3a-d72313d80f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1f987bc40c4ac0ae9104ea0f897829'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:51.895843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 942/1876] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:51.989728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 935/1877] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:51 2022] DELETE /identity/v3/users/1d1f987bc40c4ac0ae9104ea0f897829 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:52.526695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:52.702519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:52.733982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f274f-e66a-41f7-af51-dd1b2144100a service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:52.826789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:52.827660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:52.827944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:52.828220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] PATH_INFO: `/v3/projects/75466547f0554439adf0a20b38e63d3d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:52.829421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75466547f0554439adf0a20b38e63d3d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:52.831621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:52.922971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75466547f0554439adf0a20b38e63d3d', 'request_id': 'req-71f536cc-4ae1-4fc2-9be1-05d541e6f814'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:52.983959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:52.997149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75466547f0554439adf0a20b38e63d3d', 'request_id': 'req-71f536cc-4ae1-4fc2-9be1-05d541e6f814'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:53.003989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75466547f0554439adf0a20b38e63d3d', 'request_id': 'req-71f536cc-4ae1-4fc2-9be1-05d541e6f814'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:53.014199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] The token cache is being invalidate because project 75466547f0554439adf0a20b38e63d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:23:53.014488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:53.015132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71f536cc-4ae1-4fc2-9be1-05d541e6f814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:53.016688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 943/1878] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:52 2022] DELETE /identity/v3/projects/75466547f0554439adf0a20b38e63d3d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:53.021800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.022348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.022688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.041691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:53.043555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:53.105639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 936/1879] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:53.113164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.113459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.113744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.131975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:53.133196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:53.141245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 944/1880] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:53.145538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.155735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8b774-5e8a-43e5-9d78-30f90c1b8388 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.160367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.161028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.161971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.161971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.162357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:23:53.162701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.164299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.176659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 937/1881] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:23:53.180950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.190220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42c6b-e65a-418e-89bd-0135a6468924 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.194958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.195598 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.195877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.196179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.196973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.198956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.278384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 945/1882] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:23:53 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:23:53.282991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.293669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba93ec-196a-453e-8035-b80e68324850 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.348931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.349604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.349890 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.350196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.350880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.352786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.460168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 938/1883] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:23:53 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:23:53.464729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.475828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c194c53-0d2f-4ed7-9a1a-5600df292a1f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.480934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.484285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.484285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.484285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.484285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.485157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.495984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 946/1884] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:23:53.499823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.511421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da73e4-3142-4a9f-a5ec-4c393249e326 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.516410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.517103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.517382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.517656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.518406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.519978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba0203-2439-4dc7-9a2e-15324fbc0250 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.530737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 939/1885] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:23:53.534961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.545219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f354c-2046-4591-967f-7906d1c7404b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.550058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.550947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.551220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.551495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] PATH_INFO: `/v3/projects/6eb34af0f19d4eeb9646f3b02ec564d7/users/13589257fd9d433ea804f0c0cdadd277/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.566194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb34af0f19d4eeb9646f3b02ec564d7, user_id=13589257fd9d433ea804f0c0cdadd277, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.569325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.662309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 947/1886] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:23:53 2022] PUT /identity/v3/projects/6eb34af0f19d4eeb9646f3b02ec564d7/users/13589257fd9d433ea804f0c0cdadd277/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:53.671528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.671826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.672136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.703850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] MFA Rules not processed for user `13589257fd9d433ea804f0c0cdadd277`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:53.705298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:53.770100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 940/1887] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:53.782077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:53.784129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:53.842336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513b88-fef0-4b29-a34f-31f33fe9f830 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:53.846357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:53.847011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:53.847292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:53.851245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:53.856095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:53.857597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:53.866397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 948/1888] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:54.151771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:54.819468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:55.269197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:55.419208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:56.326781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:56.471097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:56.865626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:57.089826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:58.160484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:58.225677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:58.278922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:58.280670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:58.294325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b587c0d-6717-42e4-9314-83dccd222b3a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:58.299002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e586a-783d-4b26-8e1f-85f27492f376 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:58.299155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:58.300110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.300399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.300688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] PATH_INFO: `/v3/users/13589257fd9d433ea804f0c0cdadd277` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:58.302077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13589257fd9d433ea804f0c0cdadd277)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:58.304086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:58.306586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:58.307481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.307768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.308045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] PATH_INFO: `/v3/users/f575798d78294565b37bb2daf458dd21` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:58.308766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:58.309281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f575798d78294565b37bb2daf458dd21)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:58.311681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:58.501009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13589257fd9d433ea804f0c0cdadd277'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.501668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13589257fd9d433ea804f0c0cdadd277'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.502345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13589257fd9d433ea804f0c0cdadd277'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.514554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13589257fd9d433ea804f0c0cdadd277'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.521543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c12e720-7697-47b6-8cdd-48496a79c086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13589257fd9d433ea804f0c0cdadd277'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.539410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f575798d78294565b37bb2daf458dd21'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.540192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f575798d78294565b37bb2daf458dd21'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.541040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f575798d78294565b37bb2daf458dd21'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.555819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f575798d78294565b37bb2daf458dd21'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.564003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-968aefb1-1f3d-4f91-9de3-aac91fc55951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f575798d78294565b37bb2daf458dd21'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:58.617078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 941/1889] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:58 2022] DELETE /identity/v3/users/13589257fd9d433ea804f0c0cdadd277 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:58.621806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.622156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.622436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:58.643981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:58.645244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:58.695558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 949/1890] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:58 2022] DELETE /identity/v3/users/f575798d78294565b37bb2daf458dd21 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:23:58.700408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.700706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.700984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:58.714978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 942/1891] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:58.722661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:58.723900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:58.729348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:58.731438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:58.795440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 950/1892] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:23:58.811009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:58.813518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:58.842756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b174ce-d269-4111-a044-830cc90ef3b3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:58.918264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:58.918938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.919225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.919648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:58.930365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:58.932112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:58.965828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 943/1893] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:23:58.975657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46543-bfff-45b7-8ebc-27c35b5c8c84 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:58.991279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:58.991803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:58.992078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:58.992359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:59.025287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:59.029502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9af03-ef01-465d-8321-8d3324fabcf2 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:59.045442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 951/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:59.088157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:23:59.141889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:59.145991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:59.163140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aad853-da17-42f1-a164-48719d82e594 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:59.169290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:59.170267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:59.170556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:59.175800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] PATH_INFO: `/v3/users/8e69faf2109547bf8980b57e8f5bcc44` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:59.175800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e69faf2109547bf8980b57e8f5bcc44)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:59.185586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:59.387558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e69faf2109547bf8980b57e8f5bcc44'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.388217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e69faf2109547bf8980b57e8f5bcc44'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.388831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e69faf2109547bf8980b57e8f5bcc44'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.401112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e69faf2109547bf8980b57e8f5bcc44'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.408584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e69faf2109547bf8980b57e8f5bcc44'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.521190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 944/1895] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:23:59 2022] DELETE /identity/v3/users/8e69faf2109547bf8980b57e8f5bcc44 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:23:59.525513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:59.525964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:59.525964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:59.545062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:23:59.546432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:59.618225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 952/1896] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:23:59.628036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:59.629630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:59.702474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fbc0a-37fa-4dc7-841e-a33691f8334d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:59.707524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:59.708193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:59.708478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:59.708753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:59.719688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:59.721251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:59.728044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:23:59.730154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:23:59.740676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 945/1897] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:23:59.747842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a5675-fd8f-4be6-b247-8ebbdbb74382 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:23:59.753792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:23:59.754785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:23:59.755812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:23:59.756153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] PATH_INFO: `/v3/projects/2253decf5fa94603babbf5c9e8c0139f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:23:59.757473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2253decf5fa94603babbf5c9e8c0139f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:23:59.759863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:23:59.929870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2253decf5fa94603babbf5c9e8c0139f', 'request_id': 'req-433c511b-84fd-46b8-9175-f1f5bc8a47be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:23:59.995677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:00.023431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2253decf5fa94603babbf5c9e8c0139f', 'request_id': 'req-433c511b-84fd-46b8-9175-f1f5bc8a47be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.031083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2253decf5fa94603babbf5c9e8c0139f', 'request_id': 'req-433c511b-84fd-46b8-9175-f1f5bc8a47be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.044648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] The token cache is being invalidate because project 2253decf5fa94603babbf5c9e8c0139f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:00.044945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.045866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.047708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 953/1898] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:23:59 2022] DELETE /identity/v3/projects/2253decf5fa94603babbf5c9e8c0139f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:00.052606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:00.064659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28f6f7-6f6b-4059-babb-81b067780023 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:00.131772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:00.169366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:00.169366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:00.169366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] PATH_INFO: `/v3/users/5d1e41bf665647af878e8f9191efef02` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:00.169366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1e41bf665647af878e8f9191efef02)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:00.169366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:00.433758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:00.435290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:00.467326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c511b-84fd-46b8-9175-f1f5bc8a47be admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:00.467326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:00.467326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:00.467326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:00.469057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] PATH_INFO: `/v3/projects/9f66c498128c4a66832a961694744ec3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:00.469057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f66c498128c4a66832a961694744ec3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:00.469057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:00.806924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f66c498128c4a66832a961694744ec3', 'request_id': 'req-b2794aab-880e-4d72-82ae-49140d5f70dd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.807477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1e41bf665647af878e8f9191efef02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.807477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1e41bf665647af878e8f9191efef02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.807477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1e41bf665647af878e8f9191efef02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.807477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1e41bf665647af878e8f9191efef02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.807477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1e41bf665647af878e8f9191efef02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.910605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f66c498128c4a66832a961694744ec3', 'request_id': 'req-b2794aab-880e-4d72-82ae-49140d5f70dd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.918969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f66c498128c4a66832a961694744ec3', 'request_id': 'req-b2794aab-880e-4d72-82ae-49140d5f70dd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.935227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] The token cache is being invalidate because project 9f66c498128c4a66832a961694744ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:00.937841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.938566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:00.943288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 954/1899] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:00 2022] DELETE /identity/v3/projects/9f66c498128c4a66832a961694744ec3 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:00.943288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:00.943288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:00.975059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 946/1900] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:00 2022] DELETE /identity/v3/users/5d1e41bf665647af878e8f9191efef02 => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:00.975933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:00.981942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:01.096770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6776-a385-419e-aa53-fbc91c4c9891 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.103510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794aab-880e-4d72-82ae-49140d5f70dd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.116663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.121783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.121783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.121783 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] PATH_INFO: `/v3/users/50412b7b06f14cdd9394d43a4324d8ea` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.122088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.122809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.123094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.123370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] PATH_INFO: `/v3/projects/6eb34af0f19d4eeb9646f3b02ec564d7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.123986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50412b7b06f14cdd9394d43a4324d8ea)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.124652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb34af0f19d4eeb9646f3b02ec564d7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.126752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.127670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.375078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50412b7b06f14cdd9394d43a4324d8ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.375474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb34af0f19d4eeb9646f3b02ec564d7', 'request_id': 'req-57dd0d30-94b3-47e1-8552-0ecf8b5704de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.375882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50412b7b06f14cdd9394d43a4324d8ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.376482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50412b7b06f14cdd9394d43a4324d8ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.393174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50412b7b06f14cdd9394d43a4324d8ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.402215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50412b7b06f14cdd9394d43a4324d8ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.465652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb34af0f19d4eeb9646f3b02ec564d7', 'request_id': 'req-57dd0d30-94b3-47e1-8552-0ecf8b5704de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.472275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb34af0f19d4eeb9646f3b02ec564d7', 'request_id': 'req-57dd0d30-94b3-47e1-8552-0ecf8b5704de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.483643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] The token cache is being invalidate because project 6eb34af0f19d4eeb9646f3b02ec564d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:01.483934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.484567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-57dd0d30-94b3-47e1-8552-0ecf8b5704de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.486175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 955/1901] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:00 2022] DELETE /identity/v3/projects/6eb34af0f19d4eeb9646f3b02ec564d7 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:01.491718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4af-7830-4fbf-9b7f-7b64d01f7381 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.492010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4af-7830-4fbf-9b7f-7b64d01f7381 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.492290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc4af-7830-4fbf-9b7f-7b64d01f7381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.510231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dd5dc4af-7830-4fbf-9b7f-7b64d01f7381 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:01.511679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dd5dc4af-7830-4fbf-9b7f-7b64d01f7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:01.569081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 947/1902] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:00 2022] DELETE /identity/v3/users/50412b7b06f14cdd9394d43a4324d8ea => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:01.570814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:01.573450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:01.595908 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 956/1903] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:01.610452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.613058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.613754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.659656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:01.661013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:01.663849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:01.695772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f515c-70a0-4989-b073-c8247a07e592 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.724891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.728319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.728319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.728319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] PATH_INFO: `/v3/projects/856246beca554beb923fa3adfa77aa1e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.728319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=856246beca554beb923fa3adfa77aa1e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.730275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.733662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 957/1904] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:01.738633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:01.750295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5bbc1-ec6e-4ee0-aa15-f4861ce74256 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.756386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.757337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.757717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.758139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.759487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:01.759903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.761825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.780735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 958/1905] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:01.781927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:01.795406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95bbeb-efd7-44f2-9361-c3a3161238f0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.801931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.802789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.803422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.803799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.804622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.807165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.873641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856246beca554beb923fa3adfa77aa1e', 'request_id': 'req-70157f24-ea54-470c-8526-44fec0105130'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.891758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 959/1906] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:01 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:01.896710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:01.944974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856246beca554beb923fa3adfa77aa1e', 'request_id': 'req-70157f24-ea54-470c-8526-44fec0105130'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.956078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856246beca554beb923fa3adfa77aa1e', 'request_id': 'req-70157f24-ea54-470c-8526-44fec0105130'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.968718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] The token cache is being invalidate because project 856246beca554beb923fa3adfa77aa1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:01.969514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.969816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-70157f24-ea54-470c-8526-44fec0105130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:01.971670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 948/1907] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:01 2022] DELETE /identity/v3/projects/856246beca554beb923fa3adfa77aa1e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:01.977702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eca3c-ab43-4c7d-8c97-b7e656c0c88b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.978108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eca3c-ab43-4c7d-8c97-b7e656c0c88b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.978626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eca3c-ab43-4c7d-8c97-b7e656c0c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.981911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff655a50-cd1a-4ffb-bf3c-e25fb8fc46f6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:01.987778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:01.988894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:01.989364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:01.989707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:01.990543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:01.992574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:01.999649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ce3eca3c-ab43-4c7d-8c97-b7e656c0c88b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:02.007192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ce3eca3c-ab43-4c7d-8c97-b7e656c0c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.018179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 949/1908] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:02.025075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.025602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.026142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.053178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:02.054429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.066164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 950/1909] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:02.067330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.076968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04457f3-57c5-47fb-aec0-46fa699de1cc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.082551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.083582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.083871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.084455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.085208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:02.085509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.087252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.093419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 960/1910] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:01 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:02.101144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.108496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.110074 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 951/1911] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:02.114135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.126580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad3927-37b0-43b0-a205-bba63a383f42 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.130980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.131786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.132954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.133258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.134341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.137608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.146329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9b776-7c59-4f89-aeee-cf7b5e736f90 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.153271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 961/1912] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:02.155207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.162563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.163251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.163534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.165378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.166173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.166913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0332266-ea51-4ec7-ac16-578152841807 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.168164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.174990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.175999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.176502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.177107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.178850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.181832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.208111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 962/1913] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:02.209360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.233125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec9a53-a022-4654-a632-286a7b79d5ce admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.240820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.241881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.242309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.242712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] PATH_INFO: `/v3/projects/c8849f35609b4227892fa636863035d0/users/7b3bc85f100e40a28efcb3407bae86b8/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.261786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8849f35609b4227892fa636863035d0, user_id=7b3bc85f100e40a28efcb3407bae86b8, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.265234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4da8b7-3da0-4376-8de6-0c30ad2c48da admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.298685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 952/1914] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:02.302646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.356927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:02.368060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 963/1915] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:02 2022] PUT /identity/v3/projects/c8849f35609b4227892fa636863035d0/users/7b3bc85f100e40a28efcb3407bae86b8/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:02.373540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.373954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.375616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.424969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7e94-25ae-444d-92a2-02a68aec129b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.430914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] MFA Rules not processed for user `7b3bc85f100e40a28efcb3407bae86b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:02.432594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.524255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.524255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.524255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.524255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.524611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 964/1916] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:02.524708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.524708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.530991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.534258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.551252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfbf35-569c-46e9-abb1-3d734e801226 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.556713 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.557959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.558110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.558426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] PATH_INFO: `/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.559829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=911d4fe9c4364fe89ea1ca97b9a267e8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.562712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.631220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 953/1917] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/users => generated 287 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:02.632076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.633637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.674822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911d4fe9c4364fe89ea1ca97b9a267e8', 'request_id': 'req-2fd1c83e-de5c-4fad-bd19-a02bff386881'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:02.692616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec021520-67c1-441a-ae56-b276c8d37612 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.744795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911d4fe9c4364fe89ea1ca97b9a267e8', 'request_id': 'req-2fd1c83e-de5c-4fad-bd19-a02bff386881'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:02.752574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911d4fe9c4364fe89ea1ca97b9a267e8', 'request_id': 'req-2fd1c83e-de5c-4fad-bd19-a02bff386881'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:02.761713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.762614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.763078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.763351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.766707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] The token cache is being invalidate because project 911d4fe9c4364fe89ea1ca97b9a267e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:02.767068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:02.767740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:02.772914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 965/1918] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:02 2022] DELETE /identity/v3/projects/911d4fe9c4364fe89ea1ca97b9a267e8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:02.772914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.772914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.826263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.828210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faa414-23b9-4526-b4aa-614aa886707f service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.838556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 954/1919] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:02.840205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d711f2e-e028-4c1f-b4b7-cffe95b693e4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.840334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1c83e-de5c-4fad-bd19-a02bff386881 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.840784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d711f2e-e028-4c1f-b4b7-cffe95b693e4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.840986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d711f2e-e028-4c1f-b4b7-cffe95b693e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.846756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.846756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.846756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.846756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.847257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.848886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.862434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 966/1920] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:02.863950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.871946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9d711f2e-e028-4c1f-b4b7-cffe95b693e4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:02.876017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9d711f2e-e028-4c1f-b4b7-cffe95b693e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.881359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c525446-eccf-47b9-9cdc-0c5a79ed97c6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.893323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 955/1921] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:02.895418 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.896237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.896457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.897219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.897827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.898253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.899434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.900727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.903122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.916200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:02.919467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 967/1922] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:02.919580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:02.921286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.927137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 956/1923] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:02.931031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.933476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631728-daaf-4fea-b3f1-dfdb1d3acda4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.940042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.940717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3e723-5dcd-40b6-a3a8-51aacc6c8ca6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.941232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.941670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.942213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] PATH_INFO: `/v3/projects/21de054853c848a1b280fb4744a5d91b/users/c66581ae51f6424b83963eb55a56240a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.946448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.947214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.947670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.948043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.949437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:02.949840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.951716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.961523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21de054853c848a1b280fb4744a5d91b, user_id=c66581ae51f6424b83963eb55a56240a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.964793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd125d-8027-4436-ac69-36830661746a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.971128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 957/1924] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:02.971128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:02.980144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2944d-59ff-486e-97e1-121389424b0f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:02.985612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:02.986301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:02.986582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:02.986860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:02.987547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:02.989517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:02.991651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:03.077401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 968/1925] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:02 2022] PUT /identity/v3/projects/21de054853c848a1b280fb4744a5d91b/users/c66581ae51f6424b83963eb55a56240a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:03.089710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.090001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.090314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.106866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 958/1926] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:02 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:03.111777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.118000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] MFA Rules not processed for user `c66581ae51f6424b83963eb55a56240a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:03.119255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.179219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9c6d-15c2-4a94-9284-125a1eae3b99 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.186140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.186833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.187121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.192428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.192582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 969/1927] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:03.195088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.195088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.201459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.202882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.259785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72ee2c-4aaf-4638-9c96-104cfcd48dfb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.263619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.264275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.264564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.264845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.273547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.275419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.285663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 959/1928] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:03.291519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.298468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 970/1929] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:03.299565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e459c9f-954e-4af0-85ae-175a98b46d06 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.304595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.305265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.305548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.305823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.306628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.308868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.320868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.320990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 960/1930] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:03.323812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.328894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816441dc-7c0d-4290-8796-1e69f57118b3 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.333812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.334228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7893b-14dd-4ed2-8677-4de2634b8675 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.334720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.335011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.335289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] PATH_INFO: `/v3/users/c66581ae51f6424b83963eb55a56240a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.336495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66581ae51f6424b83963eb55a56240a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.339655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.339982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.340834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.341119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.341492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.342440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.344569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.355777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 961/1931] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:03.359861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.380016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24712-b2aa-4a45-b97e-8fc9dd9a791a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.387041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.387880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.388170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.388818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] PATH_INFO: `/v3/projects/57dc195cfff54f9b9ee385dc387c6079/users/74f2fd677c3c4850bdfeef936ef3b947/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.404433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57dc195cfff54f9b9ee385dc387c6079, user_id=74f2fd677c3c4850bdfeef936ef3b947, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.407167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b9329-1853-42b4-983f-74c4de352588 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.504151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 962/1932] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:03 2022] PUT /identity/v3/projects/57dc195cfff54f9b9ee385dc387c6079/users/74f2fd677c3c4850bdfeef936ef3b947/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:03.512143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.512431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.512705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.543858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] MFA Rules not processed for user `74f2fd677c3c4850bdfeef936ef3b947`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:03.545435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.549747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66581ae51f6424b83963eb55a56240a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:03.550749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66581ae51f6424b83963eb55a56240a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:03.551506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66581ae51f6424b83963eb55a56240a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:03.554106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:03.565697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66581ae51f6424b83963eb55a56240a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:03.574304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ef73f3e4-631f-4c04-95eb-01cb70d6c17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66581ae51f6424b83963eb55a56240a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:03.654304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 963/1933] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:03.657765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.659087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.675907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 971/1934] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:03 2022] DELETE /identity/v3/users/c66581ae51f6424b83963eb55a56240a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:03.681932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.683224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.683668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.709533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:03.719104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.740265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f165cb-98fb-4c3c-ba88-5cdb47109e27 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.791566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 972/1935] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:03.800790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.801404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.801520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.801759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.802059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.802754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.803046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:03.804641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.811864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:03.865237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e08d1-8bd0-462f-9943-dacdb9ba6b78 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.869172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.869829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.870185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.870468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.879881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.881324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:03.897684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 973/1936] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:03.898005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 964/1937] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:03.899911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:03.909978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbb0d-22c5-4773-afdb-9cadd47618bc service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:03.969423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:03.970222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:03.970512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:03.970817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:03.971529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:03.973459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.080052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 974/1938] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:03 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:04.081243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.092432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f466d-d00d-494c-b2b5-5447dc687470 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.097239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.098083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.098371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.098649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.099374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.100972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.111630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 965/1939] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:04.117310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.128680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc7614-ce31-4ddf-a1ca-f658ba44c7d5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.134099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.134776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.135064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.135343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.136589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.138236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde420cc-8a62-445d-9ce5-a4f321496232 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.148705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 975/1940] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:04.153520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.164419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb0346-e436-4268-83fd-7f2322bc493a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.169146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.172175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.172175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.172175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] PATH_INFO: `/v3/projects/8841a93adf404009badaf4d0f85cb1f4/users/ab7dcf7219d44f4483fd7a51304f6632/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.186610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8841a93adf404009badaf4d0f85cb1f4, user_id=ab7dcf7219d44f4483fd7a51304f6632, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.189276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.271173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 966/1941] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:04 2022] PUT /identity/v3/projects/8841a93adf404009badaf4d0f85cb1f4/users/ab7dcf7219d44f4483fd7a51304f6632/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:04.281746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.282157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.282442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.315913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] MFA Rules not processed for user `ab7dcf7219d44f4483fd7a51304f6632`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:04.315913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:04.316748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.318605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:04.378215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 976/1942] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:04.382698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.387552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a71c7a-13e4-4af4-9043-8595d99caa4a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.392414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.393287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.393568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.393913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] PATH_INFO: `/v3/projects/21de054853c848a1b280fb4744a5d91b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.395668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21de054853c848a1b280fb4744a5d91b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.396935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb23ef-7ce6-43fe-889c-3227dc842cfe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.399955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.401897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.402602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.402887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.403163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.403857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.405768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.505923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:04.517022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 977/1943] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:04.521137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.528728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21de054853c848a1b280fb4744a5d91b', 'request_id': 'req-097323b4-7120-4812-9074-63a5e6c25ea5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:04.583615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ac4fc-2021-4a13-a904-12602fda7c8a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.588231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.589078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.589358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.589636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.590355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.592271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.596959 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:04.659517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21de054853c848a1b280fb4744a5d91b', 'request_id': 'req-097323b4-7120-4812-9074-63a5e6c25ea5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:04.667971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21de054853c848a1b280fb4744a5d91b', 'request_id': 'req-097323b4-7120-4812-9074-63a5e6c25ea5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:04.680563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] The token cache is being invalidate because project 21de054853c848a1b280fb4744a5d91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:04.686191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:04.686191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-097323b4-7120-4812-9074-63a5e6c25ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:04.686191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 967/1944] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:04 2022] DELETE /identity/v3/projects/21de054853c848a1b280fb4744a5d91b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:04.686470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:04.689983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f624a-1bce-42b3-a1a9-41069820147c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.690306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f624a-1bce-42b3-a1a9-41069820147c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.690952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f624a-1bce-42b3-a1a9-41069820147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.699719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 978/1945] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:04.702047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.704420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:04.719978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c909f2-72f7-441b-8c74-3d5e74f7c664 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.721848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5f8f624a-1bce-42b3-a1a9-41069820147c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:04.723596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5f8f624a-1bce-42b3-a1a9-41069820147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:04.732257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.732991 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.733299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.733580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.734358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.750857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 968/1946] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:04.751154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.752479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.752771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.753045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.771092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 979/1947] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:04.775631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:04.775875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.777025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:04.786511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ecd24-8174-498a-a954-1ccbbe242ffa admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.786711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 969/1948] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:04.795082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.803341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.804514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.804803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.805075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] PATH_INFO: `/v3/projects/aaee4239832f4fddbd5f15039d34c2eb/users/4fefc5f34c6c4e63bd9837666c3af12a/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.812205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d4d0f-9284-4f8f-b8a7-367cb77afe55 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.817828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.818907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.819624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.820053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.821436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:04.822002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.822614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaee4239832f4fddbd5f15039d34c2eb, user_id=4fefc5f34c6c4e63bd9837666c3af12a, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.824195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.825625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.843291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 970/1949] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:24:04.843735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.855555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e093d0-a0b6-4438-8638-afd7f8bafe12 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:04.861680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:04.862928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:04.863529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:04.869206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:04.869206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:04.869206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:04.931057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 980/1950] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:04 2022] PUT /identity/v3/projects/aaee4239832f4fddbd5f15039d34c2eb/users/4fefc5f34c6c4e63bd9837666c3af12a/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:04.931492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.970057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 971/1951] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:04.973493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:04.993986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c7d5-885e-40f3-9994-abff68e9053d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.042198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e315071-5223-4042-98af-a964ec604302 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.047125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.047804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.048086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.048361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.049054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.051009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.058608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.059279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.059562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.059835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.060559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.062180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.072543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 981/1952] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:05.077559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.087425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6576-9068-4b2f-8b5f-b8997f120ebc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.092274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.093170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.093456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.093737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] PATH_INFO: `/v3/projects/aaee4239832f4fddbd5f15039d34c2eb/users/4fefc5f34c6c4e63bd9837666c3af12a/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.095803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaee4239832f4fddbd5f15039d34c2eb, user_id=4fefc5f34c6c4e63bd9837666c3af12a, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.098537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979ed8-447e-45d3-888a-386110fb5d0c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.109000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 982/1953] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:05 2022] PUT /identity/v3/projects/aaee4239832f4fddbd5f15039d34c2eb/users/4fefc5f34c6c4e63bd9837666c3af12a/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:05.119759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.120585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.120960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.128692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 972/1954] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:04 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:05.140782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.148612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] MFA Rules not processed for user `4fefc5f34c6c4e63bd9837666c3af12a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:05.149822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:05.150736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee011b3-0f1c-4c30-8eba-950374d0d5b3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.209319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:05.211412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.212160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.212649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.213022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.214557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.216286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.222627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 983/1955] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:05.229363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 973/1956] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:05.233903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.238507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.240156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:05.245163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc727e-2b38-4879-9d8f-fd680c6f7cbd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.250249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.250914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.251191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.251458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.252174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.253804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.264020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 984/1957] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:05.268250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.278541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab14d-fb35-4e28-b703-0c60e333ee60 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.283983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.285050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.285296 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.285579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] PATH_INFO: `/v3/projects/4c1fdecc420f4f60b91556535f7ff9be/users/cc48bccf62a3438d9553cc069bc8a98d/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.301314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fa7e7-f44d-4dc9-adaa-032219d227f8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.306135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1fdecc420f4f60b91556535f7ff9be, user_id=cc48bccf62a3438d9553cc069bc8a98d, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.306665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.308023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.308286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.308559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.310070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5087e-1527-4664-8c4d-0905efe244e8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.310507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:05.343990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:05.371124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.372560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.379290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 985/1958] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:05 2022] PUT /identity/v3/projects/4c1fdecc420f4f60b91556535f7ff9be/users/cc48bccf62a3438d9553cc069bc8a98d/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:05.388368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 974/1959] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:05.389975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.390295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.390573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.420941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] MFA Rules not processed for user `cc48bccf62a3438d9553cc069bc8a98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:05.423410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:05.493097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 986/1960] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:05.504766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:05.506346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:05.563673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c3d45-9f3c-498a-9d0f-45a10e187468 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:05.611820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:05.612232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:05.612518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:05.612798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:05.622232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:05.623708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c21eb5-4daa-46a5-a45e-d61f849c7e99 service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:05.632556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 975/1961] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:06.036603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:06.168940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:06.745394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:06.882334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:07.509204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:07.574579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:07.621509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:07.679528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30b27b-87ec-4bfa-8ddd-1e2e041bb1ad None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:07.683837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:07.686560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:07.686560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:07.686560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:07.688793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:07.722416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:07.756689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:07.758157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839718f9-97e2-4b6a-8534-85a7be41df2f service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:07.768227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 987/1962] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:07.802265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:07.984919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:08.321348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:08.354769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:08.365250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:08.365250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:08.365250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 976/1963] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:24:08.370679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:08.370967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:08.371266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:08.408842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:08.410422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:08.482281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:08.492026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 988/1964] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:24:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:08.506792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:08.511943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:08.536404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39271c6-a6d0-4421-8d56-2038d8be35eb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:08.540633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:08.542827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:08.547489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:08.549409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:08.549409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:08.549409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:08.559747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:08.561574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:08.573051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 977/1965] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:08.631714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2d2fe-71fe-4365-9086-a3fe2f128995 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:08.639458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:08.640350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:08.640631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:08.640913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] PATH_INFO: `/v3/users/9a58140ac84841dca6ca68a4895abc8a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:08.642185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a58140ac84841dca6ca68a4895abc8a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:08.644124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:08.875020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a58140ac84841dca6ca68a4895abc8a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:08.875740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a58140ac84841dca6ca68a4895abc8a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:08.876343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a58140ac84841dca6ca68a4895abc8a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:08.890564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a58140ac84841dca6ca68a4895abc8a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:08.897354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a58140ac84841dca6ca68a4895abc8a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.015429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 989/1966] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:08 2022] DELETE /identity/v3/users/9a58140ac84841dca6ca68a4895abc8a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:09.103812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:09.284249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:09.297827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74748f7-40a6-494f-89ce-63c64b2da9a1 service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:09.304879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:09.386416 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:09.387294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.387579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.387855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] PATH_INFO: `/v3/projects/3c4adfd8b4c74448bd28b8d0bf149c0a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.389111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4adfd8b4c74448bd28b8d0bf149c0a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:09.391087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:09.443022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:09.444546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:09.503544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f673d-942b-47f2-9ed8-164c9fe37336 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:09.523268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:09.533929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4adfd8b4c74448bd28b8d0bf149c0a', 'request_id': 'req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.573604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:09.574362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.574645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.574915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.576662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:09.602137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4adfd8b4c74448bd28b8d0bf149c0a', 'request_id': 'req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.611700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4adfd8b4c74448bd28b8d0bf149c0a', 'request_id': 'req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.637163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] The token cache is being invalidate because project 3c4adfd8b4c74448bd28b8d0bf149c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:09.637387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.638152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8d0c83d5-3394-432d-a5c2-7fb6702d9dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:09.639801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 978/1967] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:09 2022] DELETE /identity/v3/projects/3c4adfd8b4c74448bd28b8d0bf149c0a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:09.645198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.645370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.649357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.674310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:09.675154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:09.675301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f5525-bdcc-4710-84b0-26b584c7302d service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:09.675903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:09.690159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 990/1968] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:09.762522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 979/1969] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:09.768070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:09.770553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.770761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.771040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.789185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:09.790669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:09.799244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 991/1970] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:09.803427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:09.815891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241518aa-1506-46d3-ab0d-06d914345b22 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:09.823817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:09.824705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.825062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.825530 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.828232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:09.828498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:09.830540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:09.839560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:09.846054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 980/1971] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:24:09.847594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:09.857383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6815f-db27-4f4f-922b-63a7d2d0b33d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:09.862233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:09.863066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:09.863356 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:09.863629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:09.864331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:09.866267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:09.992300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 992/1972] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:09 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:09.996362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:10.006382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f17fc-6fcf-4632-9255-900bd87f749e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:10.071108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:10.071787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.072182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.072529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.073688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:10.078728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:10.116689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:10.140173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 981/1973] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:09 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:10.144168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:10.155707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a24c-1fb6-41b7-b1d0-2bb0825f01cc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:10.160994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:10.161660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.161940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.162243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.163061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:10.164656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:10.176114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 993/1974] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:10.180328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:10.186327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:10.191409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef39d7-3a08-4f23-8724-17dbd57a0384 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:10.196229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:10.196911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.197192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.197466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.198356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:10.199957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752dc69-1c06-4ad8-a9fb-1692fd3aa0dc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:10.213525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 982/1975] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:10.214982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:10.231146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dbed-9080-4b37-801c-c5d51a61b28d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:10.236325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:10.237670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.238182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.238460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] PATH_INFO: `/v3/projects/2cd90d1b84fa42109cb262e63831674e/users/8e85517a7cae4996a8a4033a8bfa9136/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.255023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd90d1b84fa42109cb262e63831674e, user_id=8e85517a7cae4996a8a4033a8bfa9136, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:10.257758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:10.379404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 994/1976] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:10 2022] PUT /identity/v3/projects/2cd90d1b84fa42109cb262e63831674e/users/8e85517a7cae4996a8a4033a8bfa9136/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:10.389848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.390251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.390604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.423680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] MFA Rules not processed for user `8e85517a7cae4996a8a4033a8bfa9136`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:10.424879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:10.497374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 983/1977] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:10.506188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:10.508009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:10.597257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09168c6d-184e-4a11-8830-1d82c68c32e9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:10.602949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:10.603591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:10.603870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:10.606154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:10.613066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:10.614680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c2e96-8361-444d-a4cf-d4293f213d48 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:10.623810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 995/1978] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:14.065719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:14.127111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d18dc-4f05-44ea-978a-46e7d3fada0f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:14.131770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:14.132581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.132858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.133137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] PATH_INFO: `/v3/users/8e85517a7cae4996a8a4033a8bfa9136` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.134374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e85517a7cae4996a8a4033a8bfa9136)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:14.136344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:14.147361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:14.291495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e85517a7cae4996a8a4033a8bfa9136'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.292610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e85517a7cae4996a8a4033a8bfa9136'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.293184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e85517a7cae4996a8a4033a8bfa9136'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.306039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e85517a7cae4996a8a4033a8bfa9136'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.314473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e85517a7cae4996a8a4033a8bfa9136'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.362567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 984/1979] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:14 2022] DELETE /identity/v3/users/8e85517a7cae4996a8a4033a8bfa9136 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:14.367093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.367572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.367664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.386355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:14.391555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.421215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:14.422704 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.435010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:14.456348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 996/1980] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:14.469937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:14.471555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.490857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dbd61-b1c3-4e5e-98dc-9347eb47eed6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:14.495495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:14.497691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.498417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.498417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] PATH_INFO: `/v3/users/7b3bc85f100e40a28efcb3407bae86b8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.501224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3bc85f100e40a28efcb3407bae86b8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:14.501485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:14.530863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5453f0-3d9a-408e-88c5-1450735281f0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:14.534562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:14.535209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.535489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.535762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.545273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:14.546865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:14.556486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 997/1981] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:14.698169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:14.699782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.714071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d9ba-5cbd-446f-ba44-c144e254dbd9 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:14.719510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:14.720470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.720714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.720985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] PATH_INFO: `/v3/users/74f2fd677c3c4850bdfeef936ef3b947` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.722217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f2fd677c3c4850bdfeef936ef3b947)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:14.722806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3bc85f100e40a28efcb3407bae86b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.723446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3bc85f100e40a28efcb3407bae86b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.724058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3bc85f100e40a28efcb3407bae86b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.724174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:14.740174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3bc85f100e40a28efcb3407bae86b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.758128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ac12a10-0e25-45dd-a32c-80b248337a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3bc85f100e40a28efcb3407bae86b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.859679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 985/1982] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:14 2022] DELETE /identity/v3/users/7b3bc85f100e40a28efcb3407bae86b8 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:14.864237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:14.864605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:14.864891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:14.876681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:14.883120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:14.884461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.945328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 986/1983] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:14.951641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f2fd677c3c4850bdfeef936ef3b947'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.952400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f2fd677c3c4850bdfeef936ef3b947'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.953085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f2fd677c3c4850bdfeef936ef3b947'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.955452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:14.956822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:14.966157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f2fd677c3c4850bdfeef936ef3b947'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:14.972769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1702a798-d4de-4119-b3d9-0a12c3cced30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f2fd677c3c4850bdfeef936ef3b947'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.016210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55165190-5a19-4968-b13c-19aad3a24d2c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.019222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.020029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.020426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.020826 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.022895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.036171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.037587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.047819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 987/1984] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:15.048605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.049881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.061273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b20cd-43ca-4070-919f-c06b0220706b service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.066149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.067956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.067956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.072287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 998/1985] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:14 2022] DELETE /identity/v3/users/74f2fd677c3c4850bdfeef936ef3b947 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:15.080112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] PATH_INFO: `/v3/projects/2cd90d1b84fa42109cb262e63831674e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.080324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.080324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.080324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.080545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cd90d1b84fa42109cb262e63831674e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.080545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.098303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:15.099560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.155123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:15.164699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 999/1986] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:15.174032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.175389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.231918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cd90d1b84fa42109cb262e63831674e', 'request_id': 'req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.238584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba3910-626a-4d87-bbf0-16255bfdeae2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.291849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.292411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.292728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.293015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.354778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cd90d1b84fa42109cb262e63831674e', 'request_id': 'req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.362360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cd90d1b84fa42109cb262e63831674e', 'request_id': 'req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.362729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.364208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee58ea-2f4f-4c8a-a1ec-7206b5764bdb service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.374415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1000/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:15.379705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] The token cache is being invalidate because project 2cd90d1b84fa42109cb262e63831674e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:15.380015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.380730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c7bff5c-0ff0-4cc1-bba1-3db0e3e20003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.383604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 988/1988] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:15 2022] DELETE /identity/v3/projects/2cd90d1b84fa42109cb262e63831674e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:15.389091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.390115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.390115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.414673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:15.414673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.428153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1001/1989] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:15.436390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427a600-d580-4b67-8287-337718c20680 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.436677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427a600-d580-4b67-8287-337718c20680 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.439190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427a600-d580-4b67-8287-337718c20680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.457035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1427a600-d580-4b67-8287-337718c20680 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:15.458474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1427a600-d580-4b67-8287-337718c20680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.469752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 989/1990] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:15.472153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.482240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506f571-a67d-4c75-acf2-7f61b0c75b70 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.487028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.487699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.487980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.488253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.488981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:15.489286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.491011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.499815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427a600-d580-4b67-8287-337718c20680 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.501210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1427a600-d580-4b67-8287-337718c20680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.520902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427a600-d580-4b67-8287-337718c20680 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.533102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.536625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1002/1991] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:24:15.538738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.539250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.539622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] PATH_INFO: `/v3/projects/c8849f35609b4227892fa636863035d0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.541518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.548731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8849f35609b4227892fa636863035d0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.549643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.555441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a7e4-8f35-4856-b16e-51b0959c76db admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.567013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.567676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.567966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.568239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.568935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.571232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.580027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:15.663544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1003/1992] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:15.665838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.671951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8849f35609b4227892fa636863035d0', 'request_id': 'req-afdc84ce-6784-4f57-9488-f4d952593f0a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.736166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971302a2-a369-4011-bbcc-6f385f8af084 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.740999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.741691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.741972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.742284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.743010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.744945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.765887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8849f35609b4227892fa636863035d0', 'request_id': 'req-afdc84ce-6784-4f57-9488-f4d952593f0a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.772302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8849f35609b4227892fa636863035d0', 'request_id': 'req-afdc84ce-6784-4f57-9488-f4d952593f0a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.784299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] The token cache is being invalidate because project c8849f35609b4227892fa636863035d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:15.784487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.785130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-afdc84ce-6784-4f57-9488-f4d952593f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:15.786821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 990/1993] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:15 2022] DELETE /identity/v3/projects/c8849f35609b4227892fa636863035d0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:15.791578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373b29-4680-4556-b005-6888008e4ffb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.791870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373b29-4680-4556-b005-6888008e4ffb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.792153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373b29-4680-4556-b005-6888008e4ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.812333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b5373b29-4680-4556-b005-6888008e4ffb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:15.813594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b5373b29-4680-4556-b005-6888008e4ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.822542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 991/1994] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:15.830792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1004/1995] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:15.832125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.832425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.832710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.839912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.842264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.851702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:15.852939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:15.856244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a158-86d1-413e-9405-9f844eff2ec6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.862122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.862856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.863148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.863422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.868551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.868739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 992/1996] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:15.868887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.872229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.880398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1005/1997] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:15.884079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.885620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9dafb-180d-4abb-ad92-23727cb75e97 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.891159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.892006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.892299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.892573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.893357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:15.897302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.897302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.897522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0f67-cd56-404d-97fb-7dd329018af1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.901864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.902728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.903205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.903561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.904386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.906238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.908649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 993/1998] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:15.913531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.921496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1006/1999] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:15.922599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:15.925200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b32d-9d36-4a35-9d37-7d75f6dbd692 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.930606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.931633 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.932368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.932368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.933096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.934256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b6f1-9ef0-464a-9797-6cd28769d04a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:15.935744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:15.939136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:15.942788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:15.942788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:15.942788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] PATH_INFO: `/v3/projects/dd6b2fe6f2c34d4e94428ad80b2595bd/users/ee992f702dff4a57a060c799c6450617/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:15.946627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:15.961361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6b2fe6f2c34d4e94428ad80b2595bd, user_id=ee992f702dff4a57a060c799c6450617, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:15.964133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.062468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 994/2000] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:15 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:16.066355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.109839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1007/2001] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:15 2022] PUT /identity/v3/projects/dd6b2fe6f2c34d4e94428ad80b2595bd/users/ee992f702dff4a57a060c799c6450617/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:16.116804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.119129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.129129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fc55-d7cf-413f-8fa1-745aa960a36d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.197357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.198342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.198473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.198653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.199385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.201320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac131d-af34-4942-8827-8813316c0d6d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.202379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a693cd0-f4c7-4f2f-964c-c5cef744f01a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.207290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.208159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.208433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.208705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] PATH_INFO: `/v3/projects/57dc195cfff54f9b9ee385dc387c6079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.210050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57dc195cfff54f9b9ee385dc387c6079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.211987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.316247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 995/2002] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:16 2022] POST /identity/v3/users => generated 285 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:16.317529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.317822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.318130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.347171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] MFA Rules not processed for user `ee992f702dff4a57a060c799c6450617`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:16.348338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.356308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57dc195cfff54f9b9ee385dc387c6079', 'request_id': 'req-90ff4af0-1895-414d-a5b1-44096e657fc3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.411677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 996/2003] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:16.412819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.426269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:16.453231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57dc195cfff54f9b9ee385dc387c6079', 'request_id': 'req-90ff4af0-1895-414d-a5b1-44096e657fc3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.460341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57dc195cfff54f9b9ee385dc387c6079', 'request_id': 'req-90ff4af0-1895-414d-a5b1-44096e657fc3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.466843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:16.473616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] The token cache is being invalidate because project 57dc195cfff54f9b9ee385dc387c6079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:16.474112 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.474835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.478067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1008/2004] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:16 2022] DELETE /identity/v3/projects/57dc195cfff54f9b9ee385dc387c6079 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:16.478067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.478682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.484823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ba9e8-aacf-440e-b450-db4273d6888a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.490047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.490718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.491563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.491563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.492150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.493778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.504070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 997/2005] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:16.505045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.506904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.519827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a385-1aab-4681-a0ee-d027e393bce2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.524590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.525468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.525748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.526197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] PATH_INFO: `/v3/users/ab7dcf7219d44f4483fd7a51304f6632` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.527884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7dcf7219d44f4483fd7a51304f6632)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.529864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.557642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4af0-1895-414d-a5b1-44096e657fc3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.563472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.564183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.564489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.564763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.571781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.577306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.578975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.587879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1009/2006] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:16.588772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.590939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.610662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ba318-a068-4978-941a-9bf58a0ada4e service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.618433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.618433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.618433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.620845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.620845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.622231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.637390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1010/2007] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:16.642711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.653127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef5b83-b831-4790-9d2b-7be3bbe6b96d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.658392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.659355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.659556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.659858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] PATH_INFO: `/v3/projects/d8f120eaf89b4ef29fc92cf544ff1000/users/3be98a16abca4b5ea9e54a29f9b4ea62/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.678408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f120eaf89b4ef29fc92cf544ff1000, user_id=3be98a16abca4b5ea9e54a29f9b4ea62, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.686797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d389d-1371-4daf-bc52-21d66559e558 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.718183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7dcf7219d44f4483fd7a51304f6632'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.718183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7dcf7219d44f4483fd7a51304f6632'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.718183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7dcf7219d44f4483fd7a51304f6632'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.731620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7dcf7219d44f4483fd7a51304f6632'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.739109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7dcf7219d44f4483fd7a51304f6632'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:16.743260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:16.764241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1011/2008] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:16 2022] PUT /identity/v3/projects/d8f120eaf89b4ef29fc92cf544ff1000/users/3be98a16abca4b5ea9e54a29f9b4ea62/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:16.782413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.782413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.782413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.791535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 998/2009] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:16 2022] DELETE /identity/v3/users/ab7dcf7219d44f4483fd7a51304f6632 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:16.818826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:16.820084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.842183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.842866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.910188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1012/2010] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:16.916869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:16.918903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:16.930907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81429b95-ab49-4a6c-90c6-6eacc4ad5086 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.935672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.936580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.936879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.937153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] PATH_INFO: `/v3/users/ee992f702dff4a57a060c799c6450617` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.938398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee992f702dff4a57a060c799c6450617)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:16.940318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:16.979063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:16.984503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4fc7-8832-4152-83e7-8a1af961fc65 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:16.988271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:16.988924 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:16.989209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:16.989486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:16.998929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:17.000586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7524616-ef02-4305-b8a8-7b454c1d43a5 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:17.010496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1013/2011] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:17.124016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee992f702dff4a57a060c799c6450617'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.125012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee992f702dff4a57a060c799c6450617'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.125630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee992f702dff4a57a060c799c6450617'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.141133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee992f702dff4a57a060c799c6450617'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.150480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee992f702dff4a57a060c799c6450617'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.251729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 999/2012] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:16 2022] DELETE /identity/v3/users/ee992f702dff4a57a060c799c6450617 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:17.256349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:17.256637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:17.257262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:17.276661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:17.277919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:17.337544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1014/2013] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:17.344909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:17.346962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:17.353359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:17.353359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:17.361909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea06cf1-b03d-4f52-9af1-462d8266e01a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:17.367573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:17.368137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:17.368533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:17.368812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] PATH_INFO: `/v3/projects/8841a93adf404009badaf4d0f85cb1f4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:17.370361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8841a93adf404009badaf4d0f85cb1f4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:17.372916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:17.423018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d769ae-b6c0-4ece-b3f5-d17313f98c42 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:17.427642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:17.428130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:17.428304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:17.428647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:17.444190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:17.445816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:17.456657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1015/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:17.489874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8841a93adf404009badaf4d0f85cb1f4', 'request_id': 'req-4a97fa11-d682-491f-bba2-47cd50d9c116'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.613566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8841a93adf404009badaf4d0f85cb1f4', 'request_id': 'req-4a97fa11-d682-491f-bba2-47cd50d9c116'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.620024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8841a93adf404009badaf4d0f85cb1f4', 'request_id': 'req-4a97fa11-d682-491f-bba2-47cd50d9c116'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.628136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:17.632950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] The token cache is being invalidate because project 8841a93adf404009badaf4d0f85cb1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:17.633239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.634489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.635867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1000/2015] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:17 2022] DELETE /identity/v3/projects/8841a93adf404009badaf4d0f85cb1f4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:17.639894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:17.651491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3cade7-866a-4be0-8c42-0d8a362bfe68 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:17.732778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:17.734124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:17.734488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:17.734830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] PATH_INFO: `/v3/users/4fefc5f34c6c4e63bd9837666c3af12a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:17.736212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fefc5f34c6c4e63bd9837666c3af12a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:17.739465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:17.935291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:17.936852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:17.952289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97fa11-d682-491f-bba2-47cd50d9c116 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:17.957614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:17.958768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:17.959126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:17.959586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] PATH_INFO: `/v3/projects/dd6b2fe6f2c34d4e94428ad80b2595bd` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:17.960951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6b2fe6f2c34d4e94428ad80b2595bd)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:17.962992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:17.974440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:17.992136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fefc5f34c6c4e63bd9837666c3af12a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.993076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fefc5f34c6c4e63bd9837666c3af12a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:17.993808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fefc5f34c6c4e63bd9837666c3af12a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.023339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fefc5f34c6c4e63bd9837666c3af12a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.031961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-40cc320f-63ee-44a9-a403-bc80610be2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fefc5f34c6c4e63bd9837666c3af12a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.110983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6b2fe6f2c34d4e94428ad80b2595bd', 'request_id': 'req-1146b12f-3682-4c74-be24-79abe01c7ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.128970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1016/2016] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:17 2022] DELETE /identity/v3/users/4fefc5f34c6c4e63bd9837666c3af12a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:18.202552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6b2fe6f2c34d4e94428ad80b2595bd', 'request_id': 'req-1146b12f-3682-4c74-be24-79abe01c7ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.212386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6b2fe6f2c34d4e94428ad80b2595bd', 'request_id': 'req-1146b12f-3682-4c74-be24-79abe01c7ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.233442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] The token cache is being invalidate because project dd6b2fe6f2c34d4e94428ad80b2595bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:18.238299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.246671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1146b12f-3682-4c74-be24-79abe01c7ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.246671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1001/2017] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:17 2022] DELETE /identity/v3/projects/dd6b2fe6f2c34d4e94428ad80b2595bd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:18.247140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.247435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.247721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.276611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:18.276611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:18.337543 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1017/2018] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:18.346058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.346712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.346712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.367557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:18.368839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:18.377822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1002/2019] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:18.382108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.391791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f2f1f-bbd8-4e84-8626-59a97053568c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.396851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.397511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.397793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.398092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.398822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:18.399126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.400879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.413103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1018/2020] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:18.417319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.427390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c437a-e83c-42a8-bc4e-2715395c6755 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.434462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.435328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.435618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.435897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.436622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.438619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.532601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1003/2021] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:18.533754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.551673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6021-7c4e-4c4e-b789-53349cbb43b2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.572606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.577717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:18.654704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.655469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.655751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.656026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.656747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.658899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.660615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3dd06-20c2-41dd-90a5-716f798bf938 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.665365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.666311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.666597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.666875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] PATH_INFO: `/v3/projects/aaee4239832f4fddbd5f15039d34c2eb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.668746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaee4239832f4fddbd5f15039d34c2eb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.673324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.760082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1019/2022] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:18 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:18.761251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.775646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9733-21d4-4ec7-953d-696db190ff33 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.783486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.784166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.784453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.788093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.788093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.788093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.799189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1020/2023] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:18.803120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.806458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaee4239832f4fddbd5f15039d34c2eb', 'request_id': 'req-8f69b422-d2e8-4f62-91b0-ce98685977a6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.812108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:18.814415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba335e-6cc3-40a8-aaeb-bfc4aa4bf14d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.826939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:18.878862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.879544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.879853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.880123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.880853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.882503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.893904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1021/2024] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:18.898116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:18.912323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e35eb-d821-4b46-b83d-1d4ac645ac13 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:18.917207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:18.918524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.918644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.919124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] PATH_INFO: `/v3/projects/8721667051ef421d8431e95445b2d83d/users/cfd4e79e95044127ac82624adc1178fb/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.934836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8721667051ef421d8431e95445b2d83d, user_id=cfd4e79e95044127ac82624adc1178fb, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:18.937372 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaee4239832f4fddbd5f15039d34c2eb', 'request_id': 'req-8f69b422-d2e8-4f62-91b0-ce98685977a6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.941410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1424b3-750e-486f-9f84-3972029df7e2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:18.943766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaee4239832f4fddbd5f15039d34c2eb', 'request_id': 'req-8f69b422-d2e8-4f62-91b0-ce98685977a6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.956180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] The token cache is being invalidate because project aaee4239832f4fddbd5f15039d34c2eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:18.960091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.960929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f69b422-d2e8-4f62-91b0-ce98685977a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:18.962660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1004/2025] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:18 2022] DELETE /identity/v3/projects/aaee4239832f4fddbd5f15039d34c2eb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:18.968682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c9cb-cf46-4b1d-80a3-3ddacebb20e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:18.969032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c9cb-cf46-4b1d-80a3-3ddacebb20e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:18.969322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c9cb-cf46-4b1d-80a3-3ddacebb20e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:18.987710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d261c9cb-cf46-4b1d-80a3-3ddacebb20e6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:18.988893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d261c9cb-cf46-4b1d-80a3-3ddacebb20e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:18.996794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1005/2026] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:19.004249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.004590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.004866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.023156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:19.024521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:19.040190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1006/2027] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:19.044837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.059725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1022/2028] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:18 2022] PUT /identity/v3/projects/8721667051ef421d8431e95445b2d83d/users/cfd4e79e95044127ac82624adc1178fb/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:19.063694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4019421-151d-44fe-aa55-35c58bb6d529 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.075932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.076697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.077588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.173006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.173755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.174066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.174349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.175272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:19.175589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.177267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.189908 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1007/2029] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:24:19.196719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.199782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] MFA Rules not processed for user `cfd4e79e95044127ac82624adc1178fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:19.203421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:19.222163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5e59-3bcc-41cf-a312-2c24cb5ba81e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.245414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.251306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.252570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.254582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.256163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.260608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.334601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1023/2030] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:19.335764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.337291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:19.353006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dda55-3bf9-4637-9202-8fdcc0feba98 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.358501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.359286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.359645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.359990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.360775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.363041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.376845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1008/2031] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/projects => generated 363 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:19.383196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.468062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1024/2032] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:19.473251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.483087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d288144-6adc-41d2-9848-d35aeb059a9f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.559907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.560824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.561180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.561523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.562751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.565584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.567778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097aaa80-fb19-4239-bace-0c7ba066183e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.572789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.573474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.574523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.575060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.575800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.580627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.651929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:19.699832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1025/2033] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:24:19.701143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.707299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1009/2034] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/users => generated 282 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:19.710819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.714388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46414875-b86b-4535-8aad-94f2cb706852 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.721683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7e410-55e9-4b0f-a365-bb2590dd038d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.723691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.724681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.724980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.725507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.726499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.730830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.733018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:19.737222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.738005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.738517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.738859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.741177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.743240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.744935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1026/2035] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:19.749990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.765772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1010/2036] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:19.770166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.772885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce717-9f23-409d-9faa-3180ae677d3d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.779069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.780205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.782675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.782675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] PATH_INFO: `/v3/projects/c19ed216238e448f9909a2ab1f7ab861/users/bb7e848fd71144c29ec358e6ce96b625/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.782863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03ebca-c1ab-4c4c-9541-aba8c1d5225c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.787973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.788773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.789114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.789587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.790358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.792077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.800624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19ed216238e448f9909a2ab1f7ab861, user_id=bb7e848fd71144c29ec358e6ce96b625, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.803415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.805715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1011/2037] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:24:19.811482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.821476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc0b-82a0-4536-99c0-779f4e92cb79 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.826443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:19.827410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.827754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.828127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] PATH_INFO: `/v3/projects/8d646f1296c44e3592cd976219dd26d1/users/8fdfaa3168a846199820d9482428c71d/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.844076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d646f1296c44e3592cd976219dd26d1, user_id=8fdfaa3168a846199820d9482428c71d, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:19.846885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a314e-a846-48b5-a8ac-d356c0006c01 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:19.893742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1027/2038] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:19 2022] PUT /identity/v3/projects/c19ed216238e448f9909a2ab1f7ab861/users/bb7e848fd71144c29ec358e6ce96b625/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:19.897620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:19.957191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1012/2039] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:19 2022] PUT /identity/v3/projects/8d646f1296c44e3592cd976219dd26d1/users/8fdfaa3168a846199820d9482428c71d/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:19.967621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:19.967937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:19.968227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:19.977019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922f11-e6a2-4030-a2cc-c0fcebcaf6d5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:19.996273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] MFA Rules not processed for user `8fdfaa3168a846199820d9482428c71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:19.997473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:20.037640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:20.038457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.038859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.039263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.040101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:20.041873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:20.052513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1028/2040] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:20.056475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:20.065960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32ed23-c345-42d7-8f9b-bc8306ab229e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:20.067557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1013/2041] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:20.070837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:20.071716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.072338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.072338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] PATH_INFO: `/v3/projects/c19ed216238e448f9909a2ab1f7ab861/users/bb7e848fd71144c29ec358e6ce96b625/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.074329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19ed216238e448f9909a2ab1f7ab861, user_id=bb7e848fd71144c29ec358e6ce96b625, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:20.077014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0d854-0d2e-4665-baa6-16c4f56d9f35 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:20.078187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:20.080440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:20.088563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1029/2042] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:20 2022] PUT /identity/v3/projects/c19ed216238e448f9909a2ab1f7ab861/users/bb7e848fd71144c29ec358e6ce96b625/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:24:20.099309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249144-635e-4599-b933-a1c8831717cb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.099671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249144-635e-4599-b933-a1c8831717cb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.100055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249144-635e-4599-b933-a1c8831717cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.128686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-23249144-635e-4599-b933-a1c8831717cb None None] MFA Rules not processed for user `bb7e848fd71144c29ec358e6ce96b625`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:20.130894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-23249144-635e-4599-b933-a1c8831717cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:20.147497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12508c-c8ac-40b8-bc67-dc12a7b43abb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:20.225554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:20.229181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.229181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.229181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.235719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1030/2043] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:20.245886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23249144-635e-4599-b933-a1c8831717cb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:20.248036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-23249144-635e-4599-b933-a1c8831717cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:20.267634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23249144-635e-4599-b933-a1c8831717cb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:20.271960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:20.273394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.273394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.273394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.285628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:20.287200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:20.300925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1031/2044] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:20.317165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:20.317165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ad95a-ccd1-439f-af95-8938659f02f2 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:20.329297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1014/2045] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:24:20.333628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:20.516576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:20.588548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:20.599218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca2081-f21c-486a-9d59-e3b7a08a5cf3 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:20.610663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:20.611347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:20.611630 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:20.611994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:20.689591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:20.691172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:20.717966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1032/2046] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:23.173991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.174762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.176057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.240351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] MFA Rules not processed for user `3be98a16abca4b5ea9e54a29f9b4ea62`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:23.241857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:23.367928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1015/2047] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:23.372586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:23.374221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:23.453843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7230-2e40-497b-a585-90dcf0ce29f1 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:23.459054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:23.459740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.460027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.460355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.461236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:23.463573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:23.532854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1033/2048] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:23 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:23.536929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:23.548257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee70db-fc66-4270-9175-fac1d3d2298a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:23.612480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:23.613280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.613573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.613850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.614581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:23.616499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:23.718187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1016/2049] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:23 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:23.722213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:23.735456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd37dc-923e-414d-9304-9bf2648e5398 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:23.740647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:23.741429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.741772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.742145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.742942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:23.744574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:23.756824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1034/2050] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:23.761239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:23.777870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142758f6-0cfc-464d-af36-3654be79eff6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:23.784671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:23.784671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.784671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.784975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] PATH_INFO: `/v3/projects/ef79770666e040ed97d751efd4af1a90/users/70385f22315446438a16fb69680419e9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.802698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79770666e040ed97d751efd4af1a90, user_id=70385f22315446438a16fb69680419e9, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:23.805406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:23.810657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:23.890641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1017/2051] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:23 2022] PUT /identity/v3/projects/ef79770666e040ed97d751efd4af1a90/users/70385f22315446438a16fb69680419e9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:23.893379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:23.903953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84ead-2089-4dee-9d43-b7731b98000a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:23.929921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:23.969291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:23.969938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:23.974291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:23.974291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:23.974291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:23.974291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb480908-799a-46a3-a31d-a0f99d5d157e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:23.984173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1035/2052] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:23.988706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:24.000917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcdc9-fcd1-4b99-88a5-9e311a4e5ed6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:24.007270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:24.008366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:24.008820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:24.009229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] PATH_INFO: `/v3/projects/ef79770666e040ed97d751efd4af1a90/users/70385f22315446438a16fb69680419e9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:24.012137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79770666e040ed97d751efd4af1a90, user_id=70385f22315446438a16fb69680419e9, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:24.015099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:24.026830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1018/2053] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:23 2022] PUT /identity/v3/projects/ef79770666e040ed97d751efd4af1a90/users/70385f22315446438a16fb69680419e9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:30.381948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:30.382590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:30.382590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:30.413350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] MFA Rules not processed for user `70385f22315446438a16fb69680419e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:30.414583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:30.487372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1036/2054] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:30.501088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:30.503229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:30.601711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9826b-79fa-4711-9c44-421c46425b6f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:30.607286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:30.607940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:30.608221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:30.608495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:30.668161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:30.669798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2fb9d-6ce0-4dda-9ac5-f711dd0725e0 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:30.683301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1019/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:31.596240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:31.598461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:31.739655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ceae4-2ebe-454e-a314-9f70c7887da3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:31.749176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:31.750417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:31.750711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:31.750998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] PATH_INFO: `/v3/users/cc48bccf62a3438d9553cc069bc8a98d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:31.752246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc48bccf62a3438d9553cc069bc8a98d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:31.918665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:32.168781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc48bccf62a3438d9553cc069bc8a98d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:32.169812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc48bccf62a3438d9553cc069bc8a98d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:32.170702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc48bccf62a3438d9553cc069bc8a98d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:32.175970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:32.196569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc48bccf62a3438d9553cc069bc8a98d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:32.218634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc48bccf62a3438d9553cc069bc8a98d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:32.299967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1037/2056] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:31 2022] DELETE /identity/v3/users/cc48bccf62a3438d9553cc069bc8a98d => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:32.305917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:32.306422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:32.306551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:32.348322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:32.350292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:32.458231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1020/2057] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:32.470280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:32.471799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:32.539257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646dc6f-d373-4763-9af5-51dbe080aea4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:32.543989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:32.545065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:32.545065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:32.545226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:32.556906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:32.558974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6da579-f67e-45f1-b67f-c950b3a5fb58 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:32.574215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1038/2058] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:33.168816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:33.169868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:33.184589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264facb-7ea3-41ca-a438-c2dec94439e6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:33.194151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:33.195030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.195494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.195769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] PATH_INFO: `/v3/projects/4c1fdecc420f4f60b91556535f7ff9be` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.197087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1fdecc420f4f60b91556535f7ff9be)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:33.200407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:33.209381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:33.369432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1fdecc420f4f60b91556535f7ff9be', 'request_id': 'req-b80d7eb6-33d2-4eb7-841c-30a834f147e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:33.424868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1fdecc420f4f60b91556535f7ff9be', 'request_id': 'req-b80d7eb6-33d2-4eb7-841c-30a834f147e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:33.431487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1fdecc420f4f60b91556535f7ff9be', 'request_id': 'req-b80d7eb6-33d2-4eb7-841c-30a834f147e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:33.444398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] The token cache is being invalidate because project 4c1fdecc420f4f60b91556535f7ff9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:33.444651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:33.445559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b80d7eb6-33d2-4eb7-841c-30a834f147e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:33.447154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1021/2059] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:33 2022] DELETE /identity/v3/projects/4c1fdecc420f4f60b91556535f7ff9be => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:33.452813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.453339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.453339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.479803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:33.481072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:33.560783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1039/2060] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:33.569002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.569431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.569713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.588382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:33.589600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:33.597506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1022/2061] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:33.601617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:33.611246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42ca2-e777-4ef3-a8d6-6c6baf76abba None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:33.616600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:33.617126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.617126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.617342 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.618227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:33.618531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:33.620841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962cbd-5c1d-483d-80e6-175091398293 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:33.638451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1040/2062] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:24:33.639442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:33.648831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ad677-db2d-4794-8e12-29e3185661e5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:33.654091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:33.654814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.655096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.655367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.656064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:33.657969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:33.861499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:33.861958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:33.862690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:33.869027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1023/2063] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:33.872392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:33.898822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] MFA Rules not processed for user `3be98a16abca4b5ea9e54a29f9b4ea62`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:33.906969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:34.014532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69522e-c4db-48d2-aa1d-fbf2ec666a79 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:34.021878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:34.022610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.022896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.023169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.023874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:34.027132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:34.095849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1041/2064] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:34.138994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1024/2065] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:33 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:34.143420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:34.150229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:34.157826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7678b-a85d-4dbb-8cfe-9595af29bb7b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:34.163350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:34.164035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.164344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.164621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.165362 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:34.167631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:34.179266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1042/2066] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:34.183751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:34.194518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54824b-b127-4685-9b29-1fafd2ab2888 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:34.199853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:34.202490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.202490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.202490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.202490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:34.204317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a78e3-592e-43dd-9e64-849ad82b6a52 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:34.221701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1025/2067] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:34.225903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:34.240560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c2cc0-8dfc-44be-9c7c-ae4078f60f6e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:34.246613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:34.247749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.248107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.248455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] PATH_INFO: `/v3/projects/dd697e7cd05c4a82a194be5957bcae06/users/e658ff0fc2384c67bf46b2af04547084/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.266442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd697e7cd05c4a82a194be5957bcae06, user_id=e658ff0fc2384c67bf46b2af04547084, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:34.269646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:34.351626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1043/2068] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:34 2022] PUT /identity/v3/projects/dd697e7cd05c4a82a194be5957bcae06/users/e658ff0fc2384c67bf46b2af04547084/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:34.365186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4111a-1800-493d-8124-da161882437b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.365186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4111a-1800-493d-8124-da161882437b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.365186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4111a-1800-493d-8124-da161882437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.391884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-74e4111a-1800-493d-8124-da161882437b None None] MFA Rules not processed for user `e658ff0fc2384c67bf46b2af04547084`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:34.393166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-74e4111a-1800-493d-8124-da161882437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:34.428736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:34.460577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1026/2069] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:34.472498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:34.475013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:34.534123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03325d3d-7c29-4299-9b06-20836f7be6d2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:34.538599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:34.539327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:34.539678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:34.540022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:34.549617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:34.554735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cb7b4-a70b-48e7-b4de-9e44d09a9e90 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:34.561720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1044/2070] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:38.697122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4111a-1800-493d-8124-da161882437b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:38.773036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4111a-1800-493d-8124-da161882437b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:38.778742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:38.779784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:38.780070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:38.781045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] PATH_INFO: `/v3/users/e658ff0fc2384c67bf46b2af04547084` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:38.781606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e658ff0fc2384c67bf46b2af04547084)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:38.783609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:38.969190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e658ff0fc2384c67bf46b2af04547084'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:38.969841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e658ff0fc2384c67bf46b2af04547084'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:38.970492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e658ff0fc2384c67bf46b2af04547084'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:38.982775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e658ff0fc2384c67bf46b2af04547084'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:38.989711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e658ff0fc2384c67bf46b2af04547084'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.063310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1027/2071] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:24:38 2022] DELETE /identity/v3/users/e658ff0fc2384c67bf46b2af04547084 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:39.068018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:39.068494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:39.068599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:39.095399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:39.096687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:39.157382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1045/2072] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:39.167476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:39.169379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:39.228943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a968d0-0b0e-41f3-a1f8-a26aacecd96d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:39.233081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:39.233899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:39.234257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:39.234539 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:39.242520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:39.246469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:39.246744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764f8b-f7c8-4984-abbc-832865351b99 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:39.256756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1028/2073] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:39.654251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:39.655663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:39.674216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd349-fbc2-4dfc-8bfb-d1ed950945ca None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:39.678337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:39.679198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:39.679396 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:39.679682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] PATH_INFO: `/v3/projects/dd697e7cd05c4a82a194be5957bcae06` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:39.680951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd697e7cd05c4a82a194be5957bcae06)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:39.682978 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:39.814557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd697e7cd05c4a82a194be5957bcae06', 'request_id': 'req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.904123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd697e7cd05c4a82a194be5957bcae06', 'request_id': 'req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.912260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd697e7cd05c4a82a194be5957bcae06', 'request_id': 'req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.916502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:39.925321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] The token cache is being invalidate because project dd697e7cd05c4a82a194be5957bcae06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:24:39.925609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.926300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a5b28e6a-4e14-443a-8dd2-8855a6b6289a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:24:39.927837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1046/2074] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:24:39 2022] DELETE /identity/v3/projects/dd697e7cd05c4a82a194be5957bcae06 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:39.933119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:39.933647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:39.933856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:39.952778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:39.954253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:40.024361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1029/2075] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:24:40.033284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.035009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.035009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.052617 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:40.053813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:40.062721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1047/2076] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:40.066928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.077852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6bb9-0427-4018-870a-a638bc2063ec None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.082888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.083555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.083840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.084113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.084848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:24:40.085157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.086792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.099975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1030/2077] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:24:40.104424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.114202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d15a4-7616-47d1-9414-ef89fe21ed18 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.119184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.121070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.121070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.121070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.121070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.123647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.201406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1048/2078] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:40 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:40.205858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.216320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38533dd-675e-4555-bbfd-6b86e71dd9d8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.272958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.273728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.274040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.274325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.275191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.277145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.350465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1031/2079] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:40 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:40.354814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.365926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ef2f-b1f1-4d9e-93ee-f6ee01550be4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.370643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.371309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.371653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.371931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.372667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.374349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.387445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1049/2080] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:40.388642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.399337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294037a4-4936-4c6f-957b-d71103449c3e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.404538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.405207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.405522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.405806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.406572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.408273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6c4ba-19e7-4ab0-88fe-e05752e48526 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.420972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1032/2081] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:40.425195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.435503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce4283-b93f-4192-b5f6-594a8724426b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.440473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.441361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.441674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.441952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] PATH_INFO: `/v3/projects/26567fb564314003bf5ae836737b990e/users/7231b60e5f9541629fec2291c547f0f4/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.457295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26567fb564314003bf5ae836737b990e, user_id=7231b60e5f9541629fec2291c547f0f4, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.460036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.561325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1050/2082] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:40 2022] PUT /identity/v3/projects/26567fb564314003bf5ae836737b990e/users/7231b60e5f9541629fec2291c547f0f4/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:40.566993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e72f17-b101-4e6e-93c2-00344a140f09 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.567294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e72f17-b101-4e6e-93c2-00344a140f09 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.567580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e72f17-b101-4e6e-93c2-00344a140f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.586630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a9e72f17-b101-4e6e-93c2-00344a140f09 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:40.587837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a9e72f17-b101-4e6e-93c2-00344a140f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:40.654236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1033/2083] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:40.664354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:40.667264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:40.732955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d12dd-dfe5-4699-aeca-482c96f179b2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:40.736637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:40.737290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:40.737564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:40.737848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:40.747195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:40.748644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:40.758279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1051/2084] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:24:47.051622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.052146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.052318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.058426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:47.089365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] MFA Rules not processed for user `7231b60e5f9541629fec2291c547f0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:47.091018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:47.178239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.179025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1034/2085] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:47.189124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5478-3328-4511-9a9e-5b8e6a82cf78 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.193955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.194671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.194962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.195237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.195949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.197894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.348705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1052/2086] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:24:47 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:47.352026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.363365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2b1c5-8ee7-4947-ba82-c76cf3ff50d5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.431475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.432196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.432527 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.432963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.433695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.435665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.517392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1035/2087] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:24:47 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:24:47.521594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.532596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd7e2-9280-4ed8-8333-0501417f4630 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.537166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.537823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.538141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.538420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.539151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.540743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.552214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1053/2088] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:47.556960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.567140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2533e60-3216-4692-b063-583af4714c69 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.571991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.572899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.573177 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.573447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] PATH_INFO: `/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1/users/f97764af4bd84f97b0c4f5c8abb3a080/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.588978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5e14b0b9834f4088b7b6ee89ab33c1, user_id=f97764af4bd84f97b0c4f5c8abb3a080, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.592230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.688578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1036/2089] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:47 2022] PUT /identity/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1/users/f97764af4bd84f97b0c4f5c8abb3a080/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:47.692795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.706756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59511fb6-b7a4-405f-bc74-9abd08584ea0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.843398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.844161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.844622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.844917 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.845735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.847463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64056b22-54c0-4923-915c-1264e3bfe533 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.849012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:47.862057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1054/2090] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:24:47.865849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:47.877015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930dc18-e9e9-46b0-9ce7-c65d7352889a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:47.882332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:47.883331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:47.883689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:47.884058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] PATH_INFO: `/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1/users/f97764af4bd84f97b0c4f5c8abb3a080/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:47.886216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5e14b0b9834f4088b7b6ee89ab33c1, user_id=f97764af4bd84f97b0c4f5c8abb3a080, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:47.890197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:47.901499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1037/2091] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:24:47 2022] PUT /identity/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1/users/f97764af4bd84f97b0c4f5c8abb3a080/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:24:53.933901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681dd3-cf25-46bf-adc4-e79a9fbf40d0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:53.934821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681dd3-cf25-46bf-adc4-e79a9fbf40d0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:53.935113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681dd3-cf25-46bf-adc4-e79a9fbf40d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:53.971053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-df681dd3-cf25-46bf-adc4-e79a9fbf40d0 None None] MFA Rules not processed for user `f97764af4bd84f97b0c4f5c8abb3a080`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:24:53.975006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-df681dd3-cf25-46bf-adc4-e79a9fbf40d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:54.074903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1055/2092] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:24:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:24:54.082964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:24:54.084695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:24:54.140072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:24:54.173814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d904f-1c34-4492-ac79-bafeaeb0b0d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:24:54.184186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:24:54.186066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:24:54.186066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:24:54.186066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:24:54.261263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:24:54.262901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:24:54.272109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1038/2093] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:24:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:04.063477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=36 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:04.267882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=36 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:10.237350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4e5-9207-457f-b37d-d6bcdfdfd9e1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:10.237820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4e5-9207-457f-b37d-d6bcdfdfd9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:10.240150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4e5-9207-457f-b37d-d6bcdfdfd9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:10.268722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4291c4e5-9207-457f-b37d-d6bcdfdfd9e1 None None] MFA Rules not processed for user `954933c1cb0249c2a86b309b6f8a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:10.268722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4291c4e5-9207-457f-b37d-d6bcdfdfd9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:10.343157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1056/2094] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:16.711817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:16.713279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:16.877516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cce5b3-030b-4b18-9b34-4714dda79ec8 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:16.883200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:16.884147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:16.884623 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:16.885185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] PATH_INFO: `/v3/users/cfd4e79e95044127ac82624adc1178fb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:16.886640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd4e79e95044127ac82624adc1178fb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:16.889451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:17.095154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd4e79e95044127ac82624adc1178fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:17.096157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd4e79e95044127ac82624adc1178fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:17.096764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd4e79e95044127ac82624adc1178fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:17.110956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd4e79e95044127ac82624adc1178fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:17.118265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd4e79e95044127ac82624adc1178fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:17.225126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1039/2095] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:16 2022] DELETE /identity/v3/users/cfd4e79e95044127ac82624adc1178fb => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:17.229371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:17.229845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:17.230158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:17.257510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:17.258709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:17.319286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1057/2096] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:17.329318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:17.330785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:17.364054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:17.430526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b9c0-4fee-429f-906f-883ca22da0db admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:17.433684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:17.434380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:17.434661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:17.434935 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:17.449754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:17.449754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:17.457105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1040/2097] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:17.970913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:17.972390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:17.998256 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a65e4-38bf-49d5-8d37-6c2af851cb96 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:18.003491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:18.004741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:18.005037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:18.005306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] PATH_INFO: `/v3/projects/8721667051ef421d8431e95445b2d83d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:18.006644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8721667051ef421d8431e95445b2d83d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:18.008569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:18.056082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:18.058285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:18.071733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43509453-d072-4422-a292-a4151b26f447 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:18.076370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:18.077771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:18.078485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:18.078778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] PATH_INFO: `/v3/users/8fdfaa3168a846199820d9482428c71d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:18.079966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdfaa3168a846199820d9482428c71d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:18.081917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:18.124988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8721667051ef421d8431e95445b2d83d', 'request_id': 'req-1452d089-58a7-4151-8a1c-dd5d5c9e505a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.200198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8721667051ef421d8431e95445b2d83d', 'request_id': 'req-1452d089-58a7-4151-8a1c-dd5d5c9e505a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.207787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8721667051ef421d8431e95445b2d83d', 'request_id': 'req-1452d089-58a7-4151-8a1c-dd5d5c9e505a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.220603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:18.231768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] The token cache is being invalidate because project 8721667051ef421d8431e95445b2d83d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:18.232293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.232793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.234563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1058/2098] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:17 2022] DELETE /identity/v3/projects/8721667051ef421d8431e95445b2d83d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:18.238346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:18.239697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:18.281741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdfaa3168a846199820d9482428c71d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.282399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdfaa3168a846199820d9482428c71d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.283234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdfaa3168a846199820d9482428c71d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.297539 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdfaa3168a846199820d9482428c71d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.304700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4dab1622-8988-4012-a87e-f57b33b47d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdfaa3168a846199820d9482428c71d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.313416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d089-58a7-4151-8a1c-dd5d5c9e505a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:18.318355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:18.319297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:18.319565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:18.319840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] PATH_INFO: `/v3/users/bb7e848fd71144c29ec358e6ce96b625` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:18.321050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7e848fd71144c29ec358e6ce96b625)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:18.323079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:18.395215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1041/2099] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:18 2022] DELETE /identity/v3/users/8fdfaa3168a846199820d9482428c71d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:25:18.404509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:18.404509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:18.404509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:18.417999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:18.419214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:18.477722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1042/2100] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:18.487728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:18.489714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:18.503762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:18.530652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7e848fd71144c29ec358e6ce96b625'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.531480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7e848fd71144c29ec358e6ce96b625'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.532478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7e848fd71144c29ec358e6ce96b625'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.555701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7e848fd71144c29ec358e6ce96b625'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.560669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c3be0-b56a-4f96-a56b-bd8eab859075 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:18.564839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7e848fd71144c29ec358e6ce96b625'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:18.565334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:18.565979 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:18.566297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:18.566567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:18.568402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:18.580934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:18.582432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:18.592938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1043/2101] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:25:18.651184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1059/2102] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:18 2022] DELETE /identity/v3/users/bb7e848fd71144c29ec358e6ce96b625 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:25:19.162996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.164635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.181388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c787a58-0acc-4490-8fdf-7a4bbe783f86 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.239371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.240858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.253591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.254635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.254896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.255337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] PATH_INFO: `/v3/projects/c19ed216238e448f9909a2ab1f7ab861` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.256401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19ed216238e448f9909a2ab1f7ab861)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.259784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.323657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd86e-2595-4aea-b153-c50a2c6419e0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.328604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.329467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.329753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.330205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] PATH_INFO: `/v3/projects/8d646f1296c44e3592cd976219dd26d1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.331297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d646f1296c44e3592cd976219dd26d1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.333217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.350934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:19.413159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19ed216238e448f9909a2ab1f7ab861', 'request_id': 'req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.463574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d646f1296c44e3592cd976219dd26d1', 'request_id': 'req-0ed9ff85-9aea-447b-b9df-7b89585b1698'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.499498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19ed216238e448f9909a2ab1f7ab861', 'request_id': 'req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.505961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19ed216238e448f9909a2ab1f7ab861', 'request_id': 'req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.517558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] The token cache is being invalidate because project c19ed216238e448f9909a2ab1f7ab861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:19.517843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.518531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-cb769b6c-5d3a-4c2f-8edf-7747c6b36e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.520061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1044/2103] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:19 2022] DELETE /identity/v3/projects/c19ed216238e448f9909a2ab1f7ab861 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:19.524542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1a56e-031e-4e74-9e43-ba78742efb95 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.524875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1a56e-031e-4e74-9e43-ba78742efb95 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.525117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1a56e-031e-4e74-9e43-ba78742efb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.566734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d646f1296c44e3592cd976219dd26d1', 'request_id': 'req-0ed9ff85-9aea-447b-b9df-7b89585b1698'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.571926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b1b1a56e-031e-4e74-9e43-ba78742efb95 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:19.577920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b1b1a56e-031e-4e74-9e43-ba78742efb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.578897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d646f1296c44e3592cd976219dd26d1', 'request_id': 'req-0ed9ff85-9aea-447b-b9df-7b89585b1698'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.586851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:19.590778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] The token cache is being invalidate because project 8d646f1296c44e3592cd976219dd26d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:19.591064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.591966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0ed9ff85-9aea-447b-b9df-7b89585b1698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:19.594867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1060/2104] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:19 2022] DELETE /identity/v3/projects/8d646f1296c44e3592cd976219dd26d1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:25:19.612062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8fda-888a-4ebe-be7f-fcce3ccb7031 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.612062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8fda-888a-4ebe-be7f-fcce3ccb7031 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.612062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8fda-888a-4ebe-be7f-fcce3ccb7031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.637082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2f6c8fda-888a-4ebe-be7f-fcce3ccb7031 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:19.638775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2f6c8fda-888a-4ebe-be7f-fcce3ccb7031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.675260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1045/2105] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:19.677067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.677620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.677894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.696843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] MFA Rules not processed for user `7231b60e5f9541629fec2291c547f0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:19.698090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.705870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1061/2106] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:19.707371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6387-5251-4c9b-8d85-c26e4a17bc37 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.707667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6387-5251-4c9b-8d85-c26e4a17bc37 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.707947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6387-5251-4c9b-8d85-c26e4a17bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.727286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-104b6387-5251-4c9b-8d85-c26e4a17bc37 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:19.728433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-104b6387-5251-4c9b-8d85-c26e4a17bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.737754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1062/2107] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:19.739124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.739408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.739744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.758883 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:19.762002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:19.771206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1046/2108] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:19.771646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1063/2109] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:19.772172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.779198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.784280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f87fc-6fdc-4a64-a88f-abf74fa7dd31 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.789144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc8cb1-fb99-4075-81fb-6dec4d5bd048 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.789502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.790203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.790486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.790898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.791623 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:25:19.792054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.793866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.793970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.798361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.810845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1064/2110] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:25:19.815139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.817906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1047/2111] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:25:19.824534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.826052 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059278b8-0050-41de-92e9-f991e055d7b9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.831450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.832154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.832625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.832911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.833607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.835551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.836018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a33268-5cef-4db2-9743-293be1155de8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:19.841477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:19.842302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:19.843211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:19.843493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:19.844194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:19.846571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:19.930001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1065/2112] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:19.931237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:19.956849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1048/2113] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:19.957990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.011304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:20.016785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0d90-bb2a-4404-9def-5d7b6f9e3716 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.062449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1467044-5c44-4801-8b91-bb319b4a08a1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.067480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.068237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.068939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.069244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.069947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.072321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.097524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.098423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.098813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.099197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.100823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.103212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.195781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1049/2114] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/users => generated 282 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:20.199768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.212359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1fd0-851f-4540-bd7c-ab7ab96e3c82 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.227223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.227915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.228454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.228740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.229482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.231267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.234036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1066/2115] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:19 2022] POST /identity/v3/users => generated 287 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:20.237801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.246611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1050/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:20.251128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.258258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2836-ace9-4f57-9441-fd43499fef1a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.258258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.258258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.258258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.259005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.260321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:20.260458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.261151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.264828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5715-8e7b-44bc-b8ea-bb92bf14e64f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.269952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.270947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.271234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.271521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.275436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1067/2117] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:25:20.275593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.276585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.279654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.291845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1051/2118] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:25:20.294749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e0275-4be2-4bf1-a7e0-278968800bbc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.296359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.302679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.303476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.303830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.304172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.305031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.307612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.312564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ffa32-6226-4e8b-b118-4300f2fc2f18 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.319734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.320759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.321042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.321320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] PATH_INFO: `/v3/projects/f98e9424caf842cbab4370ff03ac2ec5/users/fb1a198b6e904a3ca20cd826674d176b/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.321921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1068/2119] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:20.339441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.345829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98e9424caf842cbab4370ff03ac2ec5, user_id=fb1a198b6e904a3ca20cd826674d176b, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.353303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a431493-3536-41e7-aa8e-e5fa7d994deb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.363219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40db39-465d-43f9-8d52-6339c146ac4f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.371005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.371966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.372375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.372886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] PATH_INFO: `/v3/projects/cd6e338db3274331b7ab0a7370027f24/users/c7b51b4971194a23ab8a16c823843c58/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.394322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6e338db3274331b7ab0a7370027f24, user_id=c7b51b4971194a23ab8a16c823843c58, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.397413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff555bcc-de15-40bc-80e5-793f36ced535 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.435102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1052/2120] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:20 2022] PUT /identity/v3/projects/f98e9424caf842cbab4370ff03ac2ec5/users/fb1a198b6e904a3ca20cd826674d176b/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:20.445801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.446302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.446592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.479064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] MFA Rules not processed for user `fb1a198b6e904a3ca20cd826674d176b`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:20.480311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:20.488041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1069/2121] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:20 2022] PUT /identity/v3/projects/cd6e338db3274331b7ab0a7370027f24/users/c7b51b4971194a23ab8a16c823843c58/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:20.500380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.501977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.502311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.532029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] MFA Rules not processed for user `c7b51b4971194a23ab8a16c823843c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:20.532281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:20.556696 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1053/2122] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:20.560894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.624412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1070/2123] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:20.638423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.640654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:20.661334 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8bc8d-10a5-49fb-814e-90efcdfdea8f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.670323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.671280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.671746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.674949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.682573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.690385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.693567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:20.717304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac6c20-9423-4385-9ceb-dfc000dd9290 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.721287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.722209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.723061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.723061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.732431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.733951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:20.743386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1071/2124] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:25:20.841550 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1054/2125] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:20 2022] POST /identity/v3/projects => generated 365 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:20.843111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:20.859848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a3ea5-a04c-42e8-aa0d-306c34268f69 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:20.964582 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:20.965282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:20.965565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:20.965836 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:20.966559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:20.968512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.088268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1072/2126] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:20 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:21.091919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:21.117495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef86b3-25d5-4ac4-ab8c-426fcbc60344 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:21.120566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:21.121269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.121551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.121827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.122652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:21.124904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.140671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1055/2127] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:21.141456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:21.162332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07829d2-28cc-47c2-938a-97cf28f0b6fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:21.173452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:21.174555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.174903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.175187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] PATH_INFO: `/v3/projects/2c506ef791544a92a7a9a9fe912ebae4/users/f8b231f065ce4de8b1815bed91800c61/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.199735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c506ef791544a92a7a9a9fe912ebae4, user_id=f8b231f065ce4de8b1815bed91800c61, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:21.202605 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.329716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1073/2128] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:21 2022] PUT /identity/v3/projects/2c506ef791544a92a7a9a9fe912ebae4/users/f8b231f065ce4de8b1815bed91800c61/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:21.336919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:21.354085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7deba-34d0-476f-9068-09522d1f5d70 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:21.415049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:21.443943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:21.444659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.444944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.445228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.445965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:21.447613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb5ebd-cde5-40f0-a127-a0659d1bbb99 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.458487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1056/2129] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:21.462645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:21.475086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147048f-bc5b-47a8-894c-893e9ed0872a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:21.483095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:21.483993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.484266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.484540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] PATH_INFO: `/v3/projects/2c506ef791544a92a7a9a9fe912ebae4/users/f8b231f065ce4de8b1815bed91800c61/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.486559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c506ef791544a92a7a9a9fe912ebae4, user_id=f8b231f065ce4de8b1815bed91800c61, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:21.489386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.504357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1074/2130] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:21 2022] PUT /identity/v3/projects/2c506ef791544a92a7a9a9fe912ebae4/users/f8b231f065ce4de8b1815bed91800c61/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:21.516009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.516302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.516631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.545827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] MFA Rules not processed for user `f8b231f065ce4de8b1815bed91800c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:21.547273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:21.652650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1057/2131] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:21.676809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:21.778734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af47d6-dd88-4f42-93e6-5e39784a1281 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:21.785341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:21.786285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:21.786757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:21.787173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:21.799246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:21.801131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939c334-6cc4-4ee2-b0fb-115663c1fd3c service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:21.825238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1075/2132] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:22.065581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:22.077500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795056d-bb8b-44d2-a079-767ba0f43404 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:22.084860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:22.086396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:22.086945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:22.087360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:22.187858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:22.190567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5931d-4a56-4ec6-ba9c-14082112a144 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:22.212340 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1058/2133] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:22 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:22.512152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:22.512683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:22.512943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:22.532230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] MFA Rules not processed for user `c7b51b4971194a23ab8a16c823843c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:22.533443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:22.565379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:22.591964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1076/2134] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:32.199451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=76 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:33.411734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:33.791183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:33.791183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:33.791183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:33.791183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:33.791183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:33.861246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1059/2135] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:25:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:33.922485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:33.924645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:34.027921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aec09-1f40-43f0-8c11-65deca7c5286 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:34.035583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:34.036775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:34.037231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:34.047167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:34.066721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:34.068346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:34.086665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1077/2136] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:25:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:25:34.496866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:34.501055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:34.761231 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e6a0-a005-4c27-89ea-6ddb36e39b84 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:34.774100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:34.775578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:34.776140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:34.778004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:34.782281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:34.865836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:34.868504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f4765-d299-494d-abc3-1b77978a44c6 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:34.880892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1060/2137] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:36.116211 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:36.219760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94429008-0b68-43d1-9d16-9d4762a38215 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:36.231625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:36.231625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:36.231625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:36.231625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] PATH_INFO: `/v3/users/fb1a198b6e904a3ca20cd826674d176b` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:36.232087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1a198b6e904a3ca20cd826674d176b)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:36.232087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:36.457743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1a198b6e904a3ca20cd826674d176b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:36.458727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1a198b6e904a3ca20cd826674d176b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:36.459456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1a198b6e904a3ca20cd826674d176b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:36.495939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1a198b6e904a3ca20cd826674d176b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:36.518716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1a198b6e904a3ca20cd826674d176b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:36.599861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1078/2138] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:36 2022] DELETE /identity/v3/users/fb1a198b6e904a3ca20cd826674d176b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:36.601463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:36.601955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:36.602322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:36.654181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:36.659889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:36.714145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:36.753877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1061/2139] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:36.764051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:36.765905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:36.782426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:36.842762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffbb75-c264-4be0-a319-d3f7f95e8609 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:36.850368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:36.851244 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:36.852632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:36.854915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:36.870142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:36.872145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:36.888083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1079/2140] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:37.595921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:37.597400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:37.621065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2baf59-38ac-436f-aef0-1f6ede9e5be3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:37.626508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:37.628309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:37.628730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:37.628948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] PATH_INFO: `/v3/projects/f98e9424caf842cbab4370ff03ac2ec5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:37.636859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98e9424caf842cbab4370ff03ac2ec5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:37.637142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:37.801473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98e9424caf842cbab4370ff03ac2ec5', 'request_id': 'req-ab40c093-68a9-4b47-bd77-8d068602ec6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:37.880010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98e9424caf842cbab4370ff03ac2ec5', 'request_id': 'req-ab40c093-68a9-4b47-bd77-8d068602ec6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:37.887173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98e9424caf842cbab4370ff03ac2ec5', 'request_id': 'req-ab40c093-68a9-4b47-bd77-8d068602ec6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:37.912706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] The token cache is being invalidate because project f98e9424caf842cbab4370ff03ac2ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:37.913271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:37.914722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:37.917087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1062/2141] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:37 2022] DELETE /identity/v3/projects/f98e9424caf842cbab4370ff03ac2ec5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:37.921366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:37.931761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17683d-76eb-4f3d-9422-7b4615fd8055 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:38.017170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:38.018419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:38.018714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:38.018981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] PATH_INFO: `/v3/users/f8b231f065ce4de8b1815bed91800c61` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:38.020216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b231f065ce4de8b1815bed91800c61)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:38.022231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:38.245822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b231f065ce4de8b1815bed91800c61'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:38.247301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b231f065ce4de8b1815bed91800c61'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:38.248028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b231f065ce4de8b1815bed91800c61'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:38.275095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b231f065ce4de8b1815bed91800c61'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:38.305732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-20fcc276-11b8-475d-873a-d55a8927ee39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b231f065ce4de8b1815bed91800c61'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:38.430906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1080/2142] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:37 2022] DELETE /identity/v3/users/f8b231f065ce4de8b1815bed91800c61 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:39.592563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:39.594062 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:39.617942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40c093-68a9-4b47-bd77-8d068602ec6d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:39.686729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:39.688042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:39.688300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:39.688608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] PATH_INFO: `/v3/projects/2c506ef791544a92a7a9a9fe912ebae4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:39.689843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c506ef791544a92a7a9a9fe912ebae4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:39.692159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:39.835552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c506ef791544a92a7a9a9fe912ebae4', 'request_id': 'req-006cde8a-f6b3-4641-9863-2bc7c3d194c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:39.962182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c506ef791544a92a7a9a9fe912ebae4', 'request_id': 'req-006cde8a-f6b3-4641-9863-2bc7c3d194c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:39.982196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c506ef791544a92a7a9a9fe912ebae4', 'request_id': 'req-006cde8a-f6b3-4641-9863-2bc7c3d194c9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:39.982654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:40.032520 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] The token cache is being invalidate because project 2c506ef791544a92a7a9a9fe912ebae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:40.032815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:40.035502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-006cde8a-f6b3-4641-9863-2bc7c3d194c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:40.038139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1063/2143] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:39 2022] DELETE /identity/v3/projects/2c506ef791544a92a7a9a9fe912ebae4 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:40.043683 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.044788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.045294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.078309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:40.079942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:40.122571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:40.170382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1081/2144] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:40.176000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.176602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.177075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.202651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:40.204417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:40.227851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1064/2145] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:40.232106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.242572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393edae9-b34d-40dc-8598-128267bb02d5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.250639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.251294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.251963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.251963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.252619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:25:40.252936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.254765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.273348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1082/2146] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:25:40.274543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.285828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce219-8d2b-4288-ab63-b97c343cdbe9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.298116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.298116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.298116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.298116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.298691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.298691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.393871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1065/2147] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:40.395051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.406077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dca97-a092-4486-ba67-e96309a69435 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.478181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.479259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.479259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.479447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.480081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.482010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.552812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1083/2148] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:40.553941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.566048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f41-c632-476d-96f8-5a796d6469dc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.571151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.572014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.579004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.579004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.579004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.579004 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.591494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1066/2149] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:40.596168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.606542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11770237-b685-4b53-a549-0cfc06bb21c7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.611637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.612375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.612750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.613097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.613902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.615611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b947d-0256-4a89-ae3f-32a739a7e535 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.626414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1084/2150] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:40.630336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.643771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11a75-36df-4fb2-b158-61b205523a2b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.661095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.661095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.661095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.661095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] PATH_INFO: `/v3/projects/69580170773c4f99a37fdf0fc77114c9/users/5f59b22174ca4f24a4806c8c0aa3e936/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.669113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69580170773c4f99a37fdf0fc77114c9, user_id=5f59b22174ca4f24a4806c8c0aa3e936, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.672053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:40.744251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1067/2151] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:40 2022] PUT /identity/v3/projects/69580170773c4f99a37fdf0fc77114c9/users/5f59b22174ca4f24a4806c8c0aa3e936/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:40.757527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.757872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.758189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.791863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] MFA Rules not processed for user `5f59b22174ca4f24a4806c8c0aa3e936`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:40.793218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:40.883646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:40.884410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1085/2152] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:40.974634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebf180-9de6-49d2-a34e-ae51144ecd2c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:40.980185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:40.980969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:40.981363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:40.981735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:40.982948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:40.986741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.069644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1068/2153] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:40 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:41.070820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.089008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ebff-6820-4cca-8fec-f25c3fd3a50c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:41.138081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:41.197275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:41.197986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.198306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.198588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.199355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:41.202218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.305785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1086/2154] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:41 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:41.315249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.338050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099abb31-77c0-44ed-9472-838eb272b788 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:41.357147 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:41.358399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.360602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.360602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.360602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:41.361821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.386452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1069/2155] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:41.387351 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.399136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc689b9-673e-438b-b325-dea2de8957d5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:41.408180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:41.409090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.409373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.410051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] PATH_INFO: `/v3/projects/a63282a8c315414fb981968432b0263e/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.436688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63282a8c315414fb981968432b0263e, user_id=a0f3aaf0e9fd472e8b9cdc95e65c00ad, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:41.439557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.510439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1087/2156] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:41 2022] PUT /identity/v3/projects/a63282a8c315414fb981968432b0263e/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:41.515651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.528214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6abbca-9041-42c0-95d8-d1f8c74aee32 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:41.628695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:41.629853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.630332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.635929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.635929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:41.635929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207fc4c-1c0a-4fbf-9df8-8e2d3407e43e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.645576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:41.651702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1070/2157] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:41.656250 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.671487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef2b6-78a3-4c26-a12f-16b8e485db44 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:41.691083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:41.692227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.692673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.692950 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] PATH_INFO: `/v3/projects/a63282a8c315414fb981968432b0263e/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.695016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63282a8c315414fb981968432b0263e, user_id=a0f3aaf0e9fd472e8b9cdc95e65c00ad, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:41.697800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:41.718115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1088/2158] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:41 2022] PUT /identity/v3/projects/a63282a8c315414fb981968432b0263e/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:41.727999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:41.729532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:41.730437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:41.776297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] MFA Rules not processed for user `a0f3aaf0e9fd472e8b9cdc95e65c00ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:41.777647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:41.869613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1071/2159] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:41.878694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:41.880635 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:41.996312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687989e-7b59-4d98-bf0a-e3a7d1575c6a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:42.006186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:42.008942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:42.009430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:42.014134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:42.026335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:42.030482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5a456-5e78-4890-adf4-00101cf2fd32 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:42.050240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1089/2160] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:42.135363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:42.229432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebab40-664b-4f05-bf49-d5576677a35c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:42.234843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:42.236477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:42.236477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:42.236477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] PATH_INFO: `/v3/users/5f59b22174ca4f24a4806c8c0aa3e936` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:42.237875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f59b22174ca4f24a4806c8c0aa3e936)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:42.240104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:42.484998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f59b22174ca4f24a4806c8c0aa3e936'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:42.489324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f59b22174ca4f24a4806c8c0aa3e936'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:42.489324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f59b22174ca4f24a4806c8c0aa3e936'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:42.508417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f59b22174ca4f24a4806c8c0aa3e936'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:42.518849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f59b22174ca4f24a4806c8c0aa3e936'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:42.606585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1072/2161] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:42 2022] DELETE /identity/v3/users/5f59b22174ca4f24a4806c8c0aa3e936 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:42.617105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:42.617584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:42.617675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:42.641815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:42.642427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:42.721471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1090/2162] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:42.728777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:42.731125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:42.742164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:42.837735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43956d-e99a-4db6-8128-1bd5eac4a8ba admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:42.843924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:42.844971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:42.845578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:42.845864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:42.856024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:42.858128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:42.868357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1073/2163] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:43.624827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:43.627657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:43.635610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:43.655880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e11f99-218f-41a1-815f-8d9927608eb4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:43.661968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:43.663315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:43.663756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:43.664538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] PATH_INFO: `/v3/projects/69580170773c4f99a37fdf0fc77114c9` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:43.666007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69580170773c4f99a37fdf0fc77114c9)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:43.668670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:43.814987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69580170773c4f99a37fdf0fc77114c9', 'request_id': 'req-f06dbf46-a453-49bd-8447-a93e56d95583'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:43.892756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69580170773c4f99a37fdf0fc77114c9', 'request_id': 'req-f06dbf46-a453-49bd-8447-a93e56d95583'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:43.901145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69580170773c4f99a37fdf0fc77114c9', 'request_id': 'req-f06dbf46-a453-49bd-8447-a93e56d95583'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:43.915165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] The token cache is being invalidate because project 69580170773c4f99a37fdf0fc77114c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:43.916169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:43.917070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f06dbf46-a453-49bd-8447-a93e56d95583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:43.919126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1091/2164] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:43 2022] DELETE /identity/v3/projects/69580170773c4f99a37fdf0fc77114c9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:43.923190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:43.935139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a869-bcb7-4c09-a245-2ab4b7ff5e80 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:44.041414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:44.042505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:44.042888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:44.043258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] PATH_INFO: `/v3/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:44.044621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f3aaf0e9fd472e8b9cdc95e65c00ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:44.046792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:44.143316 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:44.143613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:44.143900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:44.171208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] MFA Rules not processed for user `feb66e681a894c148e3463ff5a0dd1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:44.172526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:44.250506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f3aaf0e9fd472e8b9cdc95e65c00ad'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:44.251220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f3aaf0e9fd472e8b9cdc95e65c00ad'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:44.251876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f3aaf0e9fd472e8b9cdc95e65c00ad'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:44.264700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1092/2165] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:44.275038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f3aaf0e9fd472e8b9cdc95e65c00ad'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:44.290968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-493a3c47-1dc1-47cf-a0b6-da61608c8c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f3aaf0e9fd472e8b9cdc95e65c00ad'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:44.374675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1074/2166] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:25:43 2022] DELETE /identity/v3/users/a0f3aaf0e9fd472e8b9cdc95e65c00ad => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:45.158178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:45.160580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:45.183691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4c01-a696-486a-8516-2a79062df5db None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:45.263738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:45.264627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.264909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.265184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] PATH_INFO: `/v3/projects/a63282a8c315414fb981968432b0263e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.266585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63282a8c315414fb981968432b0263e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:45.268681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:45.392128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:45.418357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63282a8c315414fb981968432b0263e', 'request_id': 'req-11025469-8d3b-4b1b-8b18-32482421e14f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:45.485670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63282a8c315414fb981968432b0263e', 'request_id': 'req-11025469-8d3b-4b1b-8b18-32482421e14f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:45.494355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63282a8c315414fb981968432b0263e', 'request_id': 'req-11025469-8d3b-4b1b-8b18-32482421e14f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:45.507589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] The token cache is being invalidate because project a63282a8c315414fb981968432b0263e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:25:45.508162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:45.509037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-11025469-8d3b-4b1b-8b18-32482421e14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:25:45.514425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1093/2167] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:25:45 2022] DELETE /identity/v3/projects/a63282a8c315414fb981968432b0263e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:45.515863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862520-dd48-4921-8110-efadfd32a95e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.516331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862520-dd48-4921-8110-efadfd32a95e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.516432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862520-dd48-4921-8110-efadfd32a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.541983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-84862520-dd48-4921-8110-efadfd32a95e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:45.544513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-84862520-dd48-4921-8110-efadfd32a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:45.573761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:45.654995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1075/2168] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:45.662380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.662926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.664765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.689059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:45.690414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:45.702953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1094/2169] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:45.703982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84862520-dd48-4921-8110-efadfd32a95e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:45.715856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84862520-dd48-4921-8110-efadfd32a95e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:45.723359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:45.724419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.724917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.725365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.726512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:25:45.726982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:45.729486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:45.753879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1076/2170] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:25:45.761938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:45.773644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb527b7e-1b6c-486a-90d9-e251a54615b2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:45.781974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:45.783012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.783402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.783757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.784464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:45.786731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:45.879597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1095/2171] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:45 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:45.882487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:45.894230 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd98e82-b8d9-484f-b496-0f6ce8beccb2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:45.971480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:45.973382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:45.973999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:45.974468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:45.978692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:45.979186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:46.070330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1077/2172] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:45 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:46.076146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:46.090539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb4b1-28ec-41f9-97bb-c649c4c46e45 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:46.100456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:46.101155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:46.101442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:46.101727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:46.102499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:46.112310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:46.136056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1096/2173] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:46.137555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:46.148014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8ab11-11ca-4b84-8c79-9c26b5e69293 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:46.164670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:46.165558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:46.165681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:46.165912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:46.166890 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:46.168504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690280c7-84b3-4440-9d16-7d5b9ff73d8f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:46.205778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1078/2174] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:46.207449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:46.230538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5187f5-5d56-4687-b9dc-9f47884dcd7a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:46.239595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:46.240514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:46.241434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:46.241723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] PATH_INFO: `/v3/projects/51ceb7333d38419baa57e6026e796012/users/2dc6d5a3727942e59fcd057a11516374/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:46.276818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ceb7333d38419baa57e6026e796012, user_id=2dc6d5a3727942e59fcd057a11516374, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:46.279615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:46.361880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1097/2175] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:46 2022] PUT /identity/v3/projects/51ceb7333d38419baa57e6026e796012/users/2dc6d5a3727942e59fcd057a11516374/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:46.369046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654747-b68c-46b1-bd51-9dae018708b1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:46.369580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654747-b68c-46b1-bd51-9dae018708b1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:46.370049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654747-b68c-46b1-bd51-9dae018708b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:46.395116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-62654747-b68c-46b1-bd51-9dae018708b1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:46.400797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-62654747-b68c-46b1-bd51-9dae018708b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:46.505251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1079/2176] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:46.514820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:46.516646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:46.594649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc61e-e7a9-4207-8f24-20f106870e68 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:46.600508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:46.601171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:46.601450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:46.601730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:46.622157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:46.623808 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:46.645134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1098/2177] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:48.563957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:48.565543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:48.565543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:48.593448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] MFA Rules not processed for user `2dc6d5a3727942e59fcd057a11516374`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:48.594691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:48.618458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:48.682154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1080/2178] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:48.684436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:48.694895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3819863-6761-495e-a96b-0b15e5adbd26 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:48.699626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:48.700280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:48.700560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:48.700850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:48.702337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:48.704666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:48.860043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1099/2179] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:25:48 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:25:48.864066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:48.877417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2559d02-3a78-4c48-8729-d03d247ed05e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:48.932881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:48.933497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:48.933765 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:48.934071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:48.934807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:48.937382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:49.017663 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1081/2180] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:25:48 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:25:49.022643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:49.045432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e644f-2543-409b-b9c1-c26faf7f0ecb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:49.052080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:49.053237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:49.053701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:49.054308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:49.055119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:49.057679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:49.071170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1100/2181] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:49.075265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:49.087562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98477-6fc8-43bd-97ae-257ae451468b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:49.092381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:49.093287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:49.093563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:49.093868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] PATH_INFO: `/v3/projects/c6d993ad2c404535b1831291a1f2f04a/users/0932e4a5a5034881b3cd859949abbd30/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:49.113815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d993ad2c404535b1831291a1f2f04a, user_id=0932e4a5a5034881b3cd859949abbd30, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:49.116857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:49.217829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1082/2182] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:49 2022] PUT /identity/v3/projects/c6d993ad2c404535b1831291a1f2f04a/users/0932e4a5a5034881b3cd859949abbd30/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:25:49.225044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:49.246571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab991-6829-47a4-b04c-4a29924a63f4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:49.277439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:49.311850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:49.312522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:49.312806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:49.313079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:49.313798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:49.315430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07085c3-9638-4d4c-99f2-a7ac1b51d654 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:49.325065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1101/2183] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:25:49.329513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:49.339875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bfbd7-f02b-4e79-9040-e24b1a1505f4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:49.345076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:49.346079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:49.346382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:49.346660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] PATH_INFO: `/v3/projects/c6d993ad2c404535b1831291a1f2f04a/users/0932e4a5a5034881b3cd859949abbd30/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:49.348699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d993ad2c404535b1831291a1f2f04a, user_id=0932e4a5a5034881b3cd859949abbd30, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:49.351387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:49.363047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1083/2184] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:25:49 2022] PUT /identity/v3/projects/c6d993ad2c404535b1831291a1f2f04a/users/0932e4a5a5034881b3cd859949abbd30/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:25:50.871317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:50.871860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:50.871985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:50.901168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] MFA Rules not processed for user `0932e4a5a5034881b3cd859949abbd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:50.902547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:50.998166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1102/2185] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:25:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:51.009237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:51.012204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:51.089985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acde78d-c5e1-4599-8058-d60a7c59ec0c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:51.102830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:51.103912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:51.104374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:51.104954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:51.161280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:51.194888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:51.196424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:51.207390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1084/2186] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:52.933031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:52.936569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:52.965358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ba418-41f7-4471-bedd-e024603d0385 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:52.975800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:52.976580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:52.976969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:52.977485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:52.991925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:52.993569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa25e16-722f-4c40-8bbe-8104c80e9f1f service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:53.005522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1103/2187] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:56.092124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:56.150063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9d4f4-7f7b-4436-b644-cfecd720bcc6 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:56.155256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:56.156118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:56.156801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:56.157094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:56.158948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:56.244332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:56.245759 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75201945-8656-48e1-a2df-923f3d33c95c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:56.255950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1085/2188] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:58.615145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:58.616212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:58.616641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:58.618488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1104/2189] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:25:58.625451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800461-d4f8-4ec4-ac25-07b53983520e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:58.626563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800461-d4f8-4ec4-ac25-07b53983520e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:58.626992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800461-d4f8-4ec4-ac25-07b53983520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:58.694124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ff800461-d4f8-4ec4-ac25-07b53983520e None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:58.695519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ff800461-d4f8-4ec4-ac25-07b53983520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:58.764276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:58.800444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1086/2190] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:25:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:58.812672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:58.815291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:58.941075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6b696-779d-4bd1-bf3c-a6f46c7cc10c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:58.945779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:58.946806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:58.946982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:58.947249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:58.982148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:58.982148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:59.000838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1105/2191] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:25:59.035600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492fc4d-d3e1-4fa1-a6d4-6e2afc185df9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.035960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492fc4d-d3e1-4fa1-a6d4-6e2afc185df9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.036200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492fc4d-d3e1-4fa1-a6d4-6e2afc185df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.144555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6492fc4d-d3e1-4fa1-a6d4-6e2afc185df9 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:59.145809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6492fc4d-d3e1-4fa1-a6d4-6e2afc185df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:59.239168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1087/2192] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:25:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:59.254854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:59.256776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:59.270910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44404eb-dabf-4d51-a64b-ad5195474e42 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.271335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44404eb-dabf-4d51-a64b-ad5195474e42 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.271837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44404eb-dabf-4d51-a64b-ad5195474e42 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.273611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1088/2193] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:25:59.279100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.279600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.280822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.306503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:59.307827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:59.321137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:25:59.323448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1089/2194] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:25:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:25:59.337303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:59.338796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:59.358269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cff23-a4bb-4288-94e8-42eda63da83e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:59.366076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:59.366757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.367039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.367318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.390214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:59.391671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5022f30-fb5c-4537-8bd8-a0344f64ac1d service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:59.418328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1090/2195] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:25:59.471863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0cd7-a6dd-4171-a6b5-ce511d6eab48 service glance] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:59.486187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:59.487070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.487293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.487421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.501307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:59.501307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a22718-125e-499d-a9b8-7c1311b85b4a service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:59.511601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1106/2196] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:25:59.523876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1e5e-12f5-438e-bd22-f899cbc07fcf None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.524312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1e5e-12f5-438e-bd22-f899cbc07fcf None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.524590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1e5e-12f5-438e-bd22-f899cbc07fcf None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.525994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1091/2197] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:25:59.534438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa52300-eaa2-4d39-b314-2cedd9d42619 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.534813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa52300-eaa2-4d39-b314-2cedd9d42619 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.535163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa52300-eaa2-4d39-b314-2cedd9d42619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.556231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2aa52300-eaa2-4d39-b314-2cedd9d42619 None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:25:59.557569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2aa52300-eaa2-4d39-b314-2cedd9d42619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:25:59.568588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1107/2198] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:25:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:25:59.580603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e1e5e-12f5-438e-bd22-f899cbc07fcf None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:25:59.592796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e1e5e-12f5-438e-bd22-f899cbc07fcf None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:25:59.596891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:25:59.597545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:25:59.597821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:25:59.598386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:25:59.611406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:25:59.612942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:25:59.638387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1092/2199] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:00.590763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d90c7-4ffc-49a5-a766-e6a37bdb6916 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:00.591277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d90c7-4ffc-49a5-a766-e6a37bdb6916 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:00.591380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d90c7-4ffc-49a5-a766-e6a37bdb6916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:00.609071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-011d90c7-4ffc-49a5-a766-e6a37bdb6916 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:00.610414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-011d90c7-4ffc-49a5-a766-e6a37bdb6916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:00.617709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1108/2200] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:26:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:00.630643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:00.642929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb5c4-88fb-4383-9192-19291c2f3dfc service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:00.647966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:00.648596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:00.648876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:00.649149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:00.657495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:00.659140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:00.667454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1093/2201] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:26:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:26:01.318515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bce29f-e913-4a99-908f-c9414f253007 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:01.318801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bce29f-e913-4a99-908f-c9414f253007 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:01.319075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bce29f-e913-4a99-908f-c9414f253007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:01.337702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-28bce29f-e913-4a99-908f-c9414f253007 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:01.338928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-28bce29f-e913-4a99-908f-c9414f253007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:01.346557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1109/2202] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:26:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:01.359052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:01.369454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587ceb-3217-44c2-869b-7ad36b64ba67 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:01.374549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:01.375191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:01.375476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:01.375755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:01.384164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:01.385736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:01.394350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1094/2203] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:26:02.040937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:02.041381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:02.041516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:02.059366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:02.061093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:02.068257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1110/2204] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:26:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:02.081193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:02.092380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a440c5f-b327-4670-a3e3-cfd02f910c49 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:02.097498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:02.098181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:02.098467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:02.098743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:02.107718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:02.109426 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:02.118322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1095/2205] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:26:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:26:04.482414 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:04.484220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:04.594099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb373-ed07-4465-8daf-97687c413d7b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:04.602887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:04.604292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:04.604292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:04.604561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:04.606906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:04.627140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:04.628765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33929f-3b79-4667-ba2a-b5dbfb3fda73 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:04.640929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1111/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:26:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:04.693313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:04.695534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:04.754351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:04.795634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b415-204e-47be-8ddc-324cc17de4e0 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:04.800778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:04.801675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:04.802000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:04.802321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] PATH_INFO: `/v3/users/c7b51b4971194a23ab8a16c823843c58` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:04.803682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b51b4971194a23ab8a16c823843c58)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:04.805593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:05.003016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b51b4971194a23ab8a16c823843c58'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.004008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b51b4971194a23ab8a16c823843c58'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.006464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b51b4971194a23ab8a16c823843c58'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.025427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b51b4971194a23ab8a16c823843c58'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.032699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b51b4971194a23ab8a16c823843c58'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.085480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1096/2207] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:26:04 2022] DELETE /identity/v3/users/c7b51b4971194a23ab8a16c823843c58 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:05.090085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:05.091344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:05.091344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:05.108599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:05.109821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:05.135952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:05.169894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1112/2208] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:05.179605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:05.181039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:05.245197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973ecc-8bdc-4243-abb3-3fb7d5f5d6cb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:05.251814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:05.251814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:05.251814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:05.251814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:05.260066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:05.261522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f927f8-f199-4bd5-834a-5f9b103ad4b9 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:05.270950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1097/2209] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:05.653148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:05.656384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:05.674594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92ae5e-54c1-4d10-9825-57f0cc472263 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:05.688601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:05.689432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:05.689718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:05.689992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] PATH_INFO: `/v3/projects/cd6e338db3274331b7ab0a7370027f24` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:05.691755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6e338db3274331b7ab0a7370027f24)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:05.693816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:05.819297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6e338db3274331b7ab0a7370027f24', 'request_id': 'req-1e6ed5a1-54d9-419c-ab68-502a93924353'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.906336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cf3-4380-4717-82fc-4e00408babdc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:05.906860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cf3-4380-4717-82fc-4e00408babdc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:05.906951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89cf3-4380-4717-82fc-4e00408babdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:05.931205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9fc89cf3-4380-4717-82fc-4e00408babdc None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:05.932709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9fc89cf3-4380-4717-82fc-4e00408babdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:05.960255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6e338db3274331b7ab0a7370027f24', 'request_id': 'req-1e6ed5a1-54d9-419c-ab68-502a93924353'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.971993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6e338db3274331b7ab0a7370027f24', 'request_id': 'req-1e6ed5a1-54d9-419c-ab68-502a93924353'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.998129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] The token cache is being invalidate because project cd6e338db3274331b7ab0a7370027f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:26:05.998129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:05.998129 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:06.009930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1113/2210] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:26:05 2022] DELETE /identity/v3/projects/cd6e338db3274331b7ab0a7370027f24 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:06.030551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1098/2211] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:26:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:06.051968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:06.051968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:06.149835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ed5a1-54d9-419c-ab68-502a93924353 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:06.156140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:06.156921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:06.157301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:06.157850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:06.168961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:06.171653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26fc1-908f-46cc-912c-093913c57eb4 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:06.182093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1114/2212] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:26:08.771091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:08.772232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:08.772516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:08.774133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1099/2213] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:26:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:26:08.781797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:08.782505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:08.782757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:08.828504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] MFA Rules not processed for user `01927a09fd62479a9640dccdda24f762`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:08.830251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:08.859750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:08.899333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1115/2214] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:26:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:08.911041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:08.913157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:08.943378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:08.954647 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8e7eb-a0fd-4491-9add-e9e0508eca94 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:08.960690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:08.965188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:08.965188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:08.965188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:08.975184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:08.977619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:08.996271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1100/2215] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:26:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:09.624884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:09.626772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:09.735240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62378a6-d196-432d-8d57-35eadb614234 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:09.741043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:09.742697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:09.743023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:09.743303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] PATH_INFO: `/v3/users/2dc6d5a3727942e59fcd057a11516374` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:09.744770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc6d5a3727942e59fcd057a11516374)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:09.746755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:09.980062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc6d5a3727942e59fcd057a11516374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:09.980872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc6d5a3727942e59fcd057a11516374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:09.981919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc6d5a3727942e59fcd057a11516374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:09.998158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc6d5a3727942e59fcd057a11516374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.005918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc6d5a3727942e59fcd057a11516374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.091136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1116/2216] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:26:09 2022] DELETE /identity/v3/users/2dc6d5a3727942e59fcd057a11516374 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:10.344140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:10.356857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670195dd-5a4d-4405-a8ed-44500419deac service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:10.421771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:10.422679 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:10.422960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:10.423234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] PATH_INFO: `/v3/projects/51ceb7333d38419baa57e6026e796012` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:10.424428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ceb7333d38419baa57e6026e796012)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:10.426370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:10.547727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ceb7333d38419baa57e6026e796012', 'request_id': 'req-252cfb95-3297-49fc-b93e-b4a6440a0e12'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.605565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ceb7333d38419baa57e6026e796012', 'request_id': 'req-252cfb95-3297-49fc-b93e-b4a6440a0e12'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.612284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ceb7333d38419baa57e6026e796012', 'request_id': 'req-252cfb95-3297-49fc-b93e-b4a6440a0e12'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.624015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] The token cache is being invalidate because project 51ceb7333d38419baa57e6026e796012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:26:10.624310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.624970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.629341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1101/2217] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:26:10 2022] DELETE /identity/v3/projects/51ceb7333d38419baa57e6026e796012 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:10.632772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:10.632772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:10.648293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf1292-65d3-44eb-ae6d-211b9f813ee7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:10.694492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:10.708474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:10.709381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:10.709670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:10.709972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] PATH_INFO: `/v3/users/0932e4a5a5034881b3cd859949abbd30` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:10.711276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0932e4a5a5034881b3cd859949abbd30)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:10.713216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:10.923785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0932e4a5a5034881b3cd859949abbd30'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.924655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0932e4a5a5034881b3cd859949abbd30'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.925319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0932e4a5a5034881b3cd859949abbd30'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.960382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0932e4a5a5034881b3cd859949abbd30'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:10.970057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa011319-e428-4c47-b186-fbd06aa1f75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0932e4a5a5034881b3cd859949abbd30'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.063344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1117/2218] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:26:10 2022] DELETE /identity/v3/users/0932e4a5a5034881b3cd859949abbd30 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:11.321075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:11.322848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:11.337005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfb95-3297-49fc-b93e-b4a6440a0e12 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:11.408193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:11.409577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.409577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.409577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] PATH_INFO: `/v3/projects/c6d993ad2c404535b1831291a1f2f04a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.410882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d993ad2c404535b1831291a1f2f04a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:11.412967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:11.439553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:11.519388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d993ad2c404535b1831291a1f2f04a', 'request_id': 'req-12110063-beab-4cac-a007-8301b70e9329'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.586972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d993ad2c404535b1831291a1f2f04a', 'request_id': 'req-12110063-beab-4cac-a007-8301b70e9329'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.593715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d993ad2c404535b1831291a1f2f04a', 'request_id': 'req-12110063-beab-4cac-a007-8301b70e9329'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.606326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] The token cache is being invalidate because project c6d993ad2c404535b1831291a1f2f04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:26:11.606691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.607352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-12110063-beab-4cac-a007-8301b70e9329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:11.609461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1102/2219] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:26:11 2022] DELETE /identity/v3/projects/c6d993ad2c404535b1831291a1f2f04a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:11.614421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.614832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.615233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.635804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:11.637237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:11.700633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1118/2220] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:11.708950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.709241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.709507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.732290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:11.734160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:11.741972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1103/2221] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:11.746676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:11.756820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b935cd-93da-40fb-a24e-37e13146fffe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:11.761657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:11.762366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.762675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.762928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.764147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:26:11.764461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:11.766135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:11.779036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1119/2222] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:26:11.784119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:11.793640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a101ddf-df1a-4cc6-8ef4-642d33e158d7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:11.798961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:11.799617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.799900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.800176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.800878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:11.803058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:11.881179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1104/2223] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:26:11 2022] POST /identity/v3/projects => generated 361 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:26:11.885229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:11.896304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe8053-332a-4a5c-9596-cb0ec781242d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:11.946130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:11.956290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:11.956894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:11.957173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:11.957443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:11.958161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:11.960076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.062816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1120/2224] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:26:11 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:26:12.066276 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:12.078796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ed308-cae6-418d-8de3-5d3ba8a71a7a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:12.084354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:12.085118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.085481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.085833 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.086682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:12.088400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.102050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1105/2225] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:26:12.105701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:12.116602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ebeaf-3f5a-41d1-87b3-5ee2979f5e17 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:12.122083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:12.122742 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.123023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.123334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.124056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:12.125691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9a104-5d97-4bd6-98a7-1d3ddfcbd0c3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.136230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1121/2226] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:26:12.142502 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:12.152904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb03ee-e193-4d4c-9d63-ce43e0ee7b8f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:12.165610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:12.166697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.166976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.167248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] PATH_INFO: `/v3/projects/27f05481fb7e4862aaed6bbe95b49be9/users/35048df91e434cd889b549a4074ee5fe/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.182203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f05481fb7e4862aaed6bbe95b49be9, user_id=35048df91e434cd889b549a4074ee5fe, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:12.184946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.291462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1106/2227] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:26:12 2022] PUT /identity/v3/projects/27f05481fb7e4862aaed6bbe95b49be9/users/35048df91e434cd889b549a4074ee5fe/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:12.302665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.302960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.303320 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.332651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] MFA Rules not processed for user `35048df91e434cd889b549a4074ee5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:12.334235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:12.405963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1122/2228] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:12.439390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:12.441169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:12.504819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8abb9-b205-4ff6-b498-2d172c7cc3f6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:12.508556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:12.509209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.509493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.509766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.518728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:12.520176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78994c27-07c7-41fd-b40c-0a9a7cc21321 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.529258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1107/2229] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:26:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:12.544522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:12.609653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db951e84-7edf-43c0-8225-94a8965c3aeb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:12.614175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:12.615035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.615313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.615584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] PATH_INFO: `/v3/users/35048df91e434cd889b549a4074ee5fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.616838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35048df91e434cd889b549a4074ee5fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:12.618870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:12.836025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35048df91e434cd889b549a4074ee5fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:12.836658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35048df91e434cd889b549a4074ee5fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:12.837349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35048df91e434cd889b549a4074ee5fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:12.848391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:12.849807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35048df91e434cd889b549a4074ee5fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:12.857104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35048df91e434cd889b549a4074ee5fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:12.925526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1123/2230] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:26:12 2022] DELETE /identity/v3/users/35048df91e434cd889b549a4074ee5fe => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:12.930274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:12.930564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:12.930840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:12.949770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:12.951050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:12.998339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:13.015860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1108/2231] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:13.022379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:13.024037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:13.084140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d471f-711d-44ae-8ec0-2d02982f045f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:13.088000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:13.088626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:13.088904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:13.089174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:13.099681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:13.101142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45517732-66f3-46ec-8623-635b1b09f16f service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:13.112163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1124/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:26:13.572410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:13.574114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:13.592836 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73421d0c-1ef7-4cbb-97be-b5d286e77881 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:13.602702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:13.603932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:13.604405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:13.604795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] PATH_INFO: `/v3/projects/27f05481fb7e4862aaed6bbe95b49be9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:13.606333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f05481fb7e4862aaed6bbe95b49be9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:13.616538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:13.715236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f05481fb7e4862aaed6bbe95b49be9', 'request_id': 'req-d71ac400-b2db-4604-befc-badbf660dc9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:13.839483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f05481fb7e4862aaed6bbe95b49be9', 'request_id': 'req-d71ac400-b2db-4604-befc-badbf660dc9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:13.847490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f05481fb7e4862aaed6bbe95b49be9', 'request_id': 'req-d71ac400-b2db-4604-befc-badbf660dc9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:13.859677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] The token cache is being invalidate because project 27f05481fb7e4862aaed6bbe95b49be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:26:13.859870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:13.860572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:26:13.872067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1109/2233] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:26:13 2022] DELETE /identity/v3/projects/27f05481fb7e4862aaed6bbe95b49be9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:26:23.106922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=33,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:23.856630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:33.164308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223677-3b27-4949-9113-5e5f7ee1c146 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:33.164838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223677-3b27-4949-9113-5e5f7ee1c146 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:33.165411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223677-3b27-4949-9113-5e5f7ee1c146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:33.185542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3c223677-3b27-4949-9113-5e5f7ee1c146 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:33.186884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3c223677-3b27-4949-9113-5e5f7ee1c146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:33.249446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1125/2234] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:33.262063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:26:33.263455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:33.342975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ac400-b2db-4604-befc-badbf660dc9c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:26:33.348594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:26:33.349251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:33.349529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:33.349799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:33.360411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:26:33.362807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a363e2b-c550-42a6-937c-2d875d2649b3 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:26:33.372505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1110/2235] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:26:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 20:26:39.117700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356265-f824-400f-8a80-a3207b4ce3ad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:26:39.118257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356265-f824-400f-8a80-a3207b4ce3ad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:26:39.118361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356265-f824-400f-8a80-a3207b4ce3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:26:39.135956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ac356265-f824-400f-8a80-a3207b4ce3ad None None] MFA Rules not processed for user `954933c1cb0249c2a86b309b6f8a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:26:39.137165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ac356265-f824-400f-8a80-a3207b4ce3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:26:39.190450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1126/2236] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:26:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:26:43.368648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=32 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:26:49.183146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=50 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:27:24.743860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012bc3-35a5-4cb6-a044-cf59016066e2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:27:24.744361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012bc3-35a5-4cb6-a044-cf59016066e2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:27:24.744361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012bc3-35a5-4cb6-a044-cf59016066e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:27:24.763286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-29012bc3-35a5-4cb6-a044-cf59016066e2 None None] MFA Rules not processed for user `0a19c51d287046578e9e4016784dad8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:27:24.764510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29012bc3-35a5-4cb6-a044-cf59016066e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:24.820567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1111/2237] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:27:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:27:27.856852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45d4c-2922-480a-a138-37850808525b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:27:27.857639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45d4c-2922-480a-a138-37850808525b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:27:27.857639 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba45d4c-2922-480a-a138-37850808525b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:27:27.875494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8ba45d4c-2922-480a-a138-37850808525b None None] MFA Rules not processed for user `c4586fdc3e8b4ee0ab58d8c236637a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:27:27.876707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8ba45d4c-2922-480a-a138-37850808525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:27.931518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1127/2238] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:27:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:27:29.468482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:27:29.469044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:27:29.469211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:27:29.489145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] MFA Rules not processed for user `feb66e681a894c148e3463ff5a0dd1be`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:27:29.490466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:29.543916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1112/2239] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:27:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:27:37.924206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=25 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:27:39.537007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=50 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:27:54.283609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:27:54.284998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:27:54.284998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:27:54.301626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] MFA Rules not processed for user `954933c1cb0249c2a86b309b6f8a334c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:27:54.302868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:54.309704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1128/2240] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:27:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:27:58.559831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:27:58.561364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:58.626672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1cc9-f162-41ac-aa5b-f7ebaa9fd4aa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:27:58.631092 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:27:58.631772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:27:58.632057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:27:58.632338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:27:58.634255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:27:58.709881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:27:58.711382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981216b-d0b6-491e-bd6e-9a11e5c10893 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:27:58.723830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1113/2241] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:27:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:00.630641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:00.632057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:00.644682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbe1ea-54aa-4407-a2d4-23b6ca90353a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:00.648318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:00.648949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:00.649226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:00.649494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:00.651209 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:00.663782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:00.665192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:00.676241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1129/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:08.717915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=54 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:10.671237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:13.917024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:13.917459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:13.917567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:13.935445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] MFA Rules not processed for user `7231b60e5f9541629fec2291c547f0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:13.936632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:13.989556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1114/2243] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:14.304136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:14.312728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf646-cc1e-4b96-a55f-276e37e27995 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:14.316498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:14.317135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:14.317419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:14.317697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:14.319501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:14.328954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:14.330935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:14.338789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1130/2244] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:14.996331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:14.997767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:15.068797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab6d62-bd65-4096-9a26-e58393527a6e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:15.073706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:15.074384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:15.074922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:15.075196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:15.076942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:15.087928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:15.089506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:15.099180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1115/2245] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:20.527725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:20.537838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931947-6f59-4108-85ad-268d3a18678c service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:20.541644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:20.542324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:20.542611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:20.542889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:20.544740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:20.555091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:20.556534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:20.564398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1131/2246] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:22.255990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:22.260727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:22.276748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5445e9-f4eb-4f12-82c5-bccebb9528aa service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:22.289707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:22.290772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:22.291228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:22.291668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:22.294221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:22.310823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:22.312423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:22.325432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1116/2247] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:24.293122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:24.306445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2114-680e-4f51-b2aa-f053d912a828 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:24.311736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:24.312386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:24.312667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:24.312944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:24.314697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:24.325375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:24.326865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:24.336001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1132/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:28.332042 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:28.333431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:28.346864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d602bc3-e776-475b-a6bf-3280f23e0606 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:28.352369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:28.353001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:28.353286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:28.353571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:28.355299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:28.365654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:28.367250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:28.375945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1117/2249] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:31.036434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:31.047988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069b194-dc63-484a-9c16-3ab4d6ae330c service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:31.053833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:31.054504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:31.054784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:31.055069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:31.056919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:31.068492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:31.070198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:31.079585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1133/2250] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:32.172986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:32.184400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4a8b5-1d08-4803-8c98-a08e057ea67b service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:32.190580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:32.191234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:32.191517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:32.191792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:32.193709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:32.207252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:32.208824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:32.220545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1118/2251] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:38.993800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:39.005453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b744-927b-4c8b-a744-99720d53ef1d service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:39.010914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:39.011545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:39.011834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:39.012101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:39.013848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:39.025686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:39.027308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:39.037655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1134/2252] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:41.146809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:41.154846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=76 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:41.156577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f209c8-d631-4952-bec7-f7ac4a97fde3 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:41.160825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:41.161460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:41.161748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:41.162039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:41.163816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:41.235815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:41.237382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:41.247883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1119/2253] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:45.731193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:45.733047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:45.751479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=31 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:45.751479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded8039-a11e-4f2d-b771-f36b080f370a service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:45.753667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:45.758982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:45.759275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:45.759560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] PATH_INFO: `/v3/users/feb66e681a894c148e3463ff5a0dd1be` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:45.761050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb66e681a894c148e3463ff5a0dd1be)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:45.764731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:46.010233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb66e681a894c148e3463ff5a0dd1be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.010816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb66e681a894c148e3463ff5a0dd1be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.011411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb66e681a894c148e3463ff5a0dd1be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.025662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb66e681a894c148e3463ff5a0dd1be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.033131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-48e40226-6215-41cd-8a3e-cdc02f938d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb66e681a894c148e3463ff5a0dd1be'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.110600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1135/2254] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:28:45 2022] DELETE /identity/v3/users/feb66e681a894c148e3463ff5a0dd1be => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:46.368872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:46.379182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81961-dfe5-4cd9-9716-f1011a4650e0 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:46.436342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:46.437201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.437479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.437773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] PATH_INFO: `/v3/projects/02969314f19c4dc182b0dbe76aad9a34` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.438977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02969314f19c4dc182b0dbe76aad9a34)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:46.440874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:46.538782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02969314f19c4dc182b0dbe76aad9a34', 'request_id': 'req-666772fc-c7fa-4667-bf8c-afdc05695d77'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.613440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02969314f19c4dc182b0dbe76aad9a34', 'request_id': 'req-666772fc-c7fa-4667-bf8c-afdc05695d77'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.619405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02969314f19c4dc182b0dbe76aad9a34', 'request_id': 'req-666772fc-c7fa-4667-bf8c-afdc05695d77'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.631096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] The token cache is being invalidate because project 02969314f19c4dc182b0dbe76aad9a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:28:46.631381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.632007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-666772fc-c7fa-4667-bf8c-afdc05695d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:46.633584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1120/2255] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:28:46 2022] DELETE /identity/v3/projects/02969314f19c4dc182b0dbe76aad9a34 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:46.644538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.644830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.645108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.662911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:46.664110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:46.721236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1136/2256] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:46.729072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.729567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.729854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.747526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:46.748721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:46.757056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1121/2257] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:46.761081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:46.771172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06d5c9-f42d-451d-878e-25546643814d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:46.775727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:46.776379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.776662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.776944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.777669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:28:46.777976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:46.779596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:46.791214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1137/2258] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:28:46.795210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:46.804399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7289819-7d95-4b8a-b622-19feb1fda083 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:46.809085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:46.809727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.810005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.810310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.810991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:46.812871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:46.870819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:46.878139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1122/2259] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:28:46 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:28:46.882334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:46.893219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5082fe1-0abd-44d0-8258-4e16704df5a7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:46.949704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:46.950393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:46.950675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:46.950949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:46.951640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:46.953555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:46.961572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:47.082460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1138/2260] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:28:46 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:28:47.086568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.097244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b8616-533e-4ea5-9f88-ac77bdbe4d00 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.101856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.102561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.102841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.103112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.103902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.105491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.116797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1123/2261] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:47.120919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.131618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97750c-66ad-4bcf-aaca-59639f3c4a69 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.136610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.137468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.137568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.137822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.138586 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.140277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505ab1f-8285-4014-9caa-7ade7d570024 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.154001 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1139/2262] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:47.155236 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.165438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adc72a-1cab-4098-b497-0401b79761eb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.169977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.170886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.171164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.171438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] PATH_INFO: `/v3/projects/aa1baaf8a0424dc4a32aef7695018d9b/users/e49f9cfbf1bf492e8786b538f0b8c7fb/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.186302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1baaf8a0424dc4a32aef7695018d9b, user_id=e49f9cfbf1bf492e8786b538f0b8c7fb, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.189076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.253281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1124/2263] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:47 2022] PUT /identity/v3/projects/aa1baaf8a0424dc4a32aef7695018d9b/users/e49f9cfbf1bf492e8786b538f0b8c7fb/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:47.263775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.264068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.265804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.290895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] MFA Rules not processed for user `e49f9cfbf1bf492e8786b538f0b8c7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:47.292169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:47.355349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1140/2264] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:47.359709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.419307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22119d7b-3eae-4790-b865-cadfc6a07cab admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.423970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.424615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.424895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.425170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.426501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.428381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.511388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1125/2265] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:28:47.515321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.525678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400408d-cea8-4a15-b86c-d04d585f1c72 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.580530 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.581193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.581481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.581758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.582484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.584519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.659245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1141/2266] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:28:47 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:28:47.674144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.684982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f4c09-22a1-4f8e-8eac-aa584bb391ce admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.689635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.690336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.690622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.690903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.691659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.693255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.703954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1126/2267] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:47.708273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.717980 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346a6-023f-460d-a37d-980b3b057872 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.723172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.724031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.724309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.724577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] PATH_INFO: `/v3/projects/1237cf16b050403cadb0c4fb51e5fadc/users/6521fbc686f746778d2c7519aac2d44d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.739103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1237cf16b050403cadb0c4fb51e5fadc, user_id=6521fbc686f746778d2c7519aac2d44d, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.742047 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.795731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1142/2268] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:47 2022] PUT /identity/v3/projects/1237cf16b050403cadb0c4fb51e5fadc/users/6521fbc686f746778d2c7519aac2d44d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:47.799616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.809824 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab92a9-ba60-435e-868b-115ba975922c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.868275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.868933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.869213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.869486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.870234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.871817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64263e-a774-482d-ac81-e7521d03a5ee admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.881119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1127/2269] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:47.885292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:47.895877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22a374-b194-4204-a397-4b60c5c9efae admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:47.900439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:47.901314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.901613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.901875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] PATH_INFO: `/v3/projects/1237cf16b050403cadb0c4fb51e5fadc/users/6521fbc686f746778d2c7519aac2d44d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.903861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1237cf16b050403cadb0c4fb51e5fadc, user_id=6521fbc686f746778d2c7519aac2d44d, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:47.906572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:47.917176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1143/2270] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:47 2022] PUT /identity/v3/projects/1237cf16b050403cadb0c4fb51e5fadc/users/6521fbc686f746778d2c7519aac2d44d/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:47.928114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:47.928438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:47.928721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:47.954589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] MFA Rules not processed for user `6521fbc686f746778d2c7519aac2d44d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:47.955786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:47.975916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:48.022323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1128/2271] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:48.033324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:48.035335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:48.061706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:48.094662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745388d-1da1-4e12-b6d5-9cf3ad78024b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:48.099222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:48.099222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:48.100781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:48.100781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:48.108896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:48.111095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4ad7-eb75-468f-8c69-61547a28b305 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:48.120620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1144/2272] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:48.881654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:48.941666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd4fb7-4908-493f-a453-b12b4a1a0a34 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:48.946697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:48.947571 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:48.947855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:48.948135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] PATH_INFO: `/v3/users/e49f9cfbf1bf492e8786b538f0b8c7fb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:48.949606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49f9cfbf1bf492e8786b538f0b8c7fb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:48.951566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:49.111879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49f9cfbf1bf492e8786b538f0b8c7fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.112480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49f9cfbf1bf492e8786b538f0b8c7fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.113075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49f9cfbf1bf492e8786b538f0b8c7fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.124446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49f9cfbf1bf492e8786b538f0b8c7fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.131578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49f9cfbf1bf492e8786b538f0b8c7fb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.178939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1129/2273] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:28:48 2022] DELETE /identity/v3/users/e49f9cfbf1bf492e8786b538f0b8c7fb => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:49.183449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:49.184060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:49.184060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:49.202412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:49.203717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:49.268277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1145/2274] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:49.278146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:49.279572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:49.322670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:49.337984 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658af8-4f04-4b8f-bea1-cee785059363 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:49.341687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:49.342360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:49.342645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:49.342919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:49.352960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:49.354421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:49.365432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1130/2275] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:49.798648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:49.800111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:49.814653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a860a3-32c2-4ef2-8681-07de925f7b1c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:49.820723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:49.821607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:49.821893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:49.822303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] PATH_INFO: `/v3/projects/aa1baaf8a0424dc4a32aef7695018d9b` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:49.823567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1baaf8a0424dc4a32aef7695018d9b)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:49.825858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:49.939218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1baaf8a0424dc4a32aef7695018d9b', 'request_id': 'req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.978822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1baaf8a0424dc4a32aef7695018d9b', 'request_id': 'req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.985050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1baaf8a0424dc4a32aef7695018d9b', 'request_id': 'req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.995258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] The token cache is being invalidate because project aa1baaf8a0424dc4a32aef7695018d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:28:49.995541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.996159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:49.997703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1146/2276] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:28:49 2022] DELETE /identity/v3/projects/aa1baaf8a0424dc4a32aef7695018d9b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:50.001612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:50.011208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7426611-5d41-4a73-96f2-18e6b922e1cf service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:50.067675 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:50.068552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:50.068928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:50.069191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] PATH_INFO: `/v3/users/6521fbc686f746778d2c7519aac2d44d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:50.070396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6521fbc686f746778d2c7519aac2d44d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:50.072454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:50.244518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6521fbc686f746778d2c7519aac2d44d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:50.245165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6521fbc686f746778d2c7519aac2d44d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:50.245819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6521fbc686f746778d2c7519aac2d44d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:50.257982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6521fbc686f746778d2c7519aac2d44d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:50.264892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e3783a-7d85-438a-93ac-488a043429cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6521fbc686f746778d2c7519aac2d44d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:50.321159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1131/2277] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:28:50 2022] DELETE /identity/v3/users/6521fbc686f746778d2c7519aac2d44d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:50.752463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:50.754007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:50.769332 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffb3e-ee5b-46bd-9238-f797f9fd817e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:50.846398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:50.858843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:50.859589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:50.859917 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:50.860203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] PATH_INFO: `/v3/projects/1237cf16b050403cadb0c4fb51e5fadc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:50.871584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1237cf16b050403cadb0c4fb51e5fadc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:50.871584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.071929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1237cf16b050403cadb0c4fb51e5fadc', 'request_id': 'req-1b0f8122-3540-4a54-b86d-5d0de8c79052'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:51.133248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1237cf16b050403cadb0c4fb51e5fadc', 'request_id': 'req-1b0f8122-3540-4a54-b86d-5d0de8c79052'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:51.140667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1237cf16b050403cadb0c4fb51e5fadc', 'request_id': 'req-1b0f8122-3540-4a54-b86d-5d0de8c79052'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:51.151443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] The token cache is being invalidate because project 1237cf16b050403cadb0c4fb51e5fadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:28:51.151729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:51.152381 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1b0f8122-3540-4a54-b86d-5d0de8c79052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:51.153916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1147/2278] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:28:50 2022] DELETE /identity/v3/projects/1237cf16b050403cadb0c4fb51e5fadc => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:51.158741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.159243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.159349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.177325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:51.178641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:51.231434 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:51.239972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1132/2279] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:51.247740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13676-b916-4c21-8628-18645fba008e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.248026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13676-b916-4c21-8628-18645fba008e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.248302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13676-b916-4c21-8628-18645fba008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.266948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-28f13676-b916-4c21-8628-18645fba008e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:51.268140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-28f13676-b916-4c21-8628-18645fba008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:51.279182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1148/2280] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:51.285907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.290936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582a9f2-ae2c-46bf-a50d-b4d4a4f93322 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.296365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.297213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.297512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.297793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.298591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:28:51.298906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.300590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.312441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1133/2281] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:28:51.316838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f13676-b916-4c21-8628-18645fba008e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.327041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f13676-b916-4c21-8628-18645fba008e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.331887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.333482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.333769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.334075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.334789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.336763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.450451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1149/2282] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:28:51.454371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.464135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e3c3-b9d4-4f8d-98cf-9c7df3eefa7b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.519447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.520140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.520443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.520731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.521435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.523404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.583975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1134/2283] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:28:51.593460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.603732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6cac-71e7-49e6-982e-3e6cb8a6936b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.608492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.609151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.610078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.610364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.611092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.612725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.623469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1150/2284] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:51.627097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.637586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c7f7-d0e6-4825-9550-f5473c692fbf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.642226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.642875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.643162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.643440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.644166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.645745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b980bc-03ea-41df-ae40-01f070e5f4c2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.655528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1135/2285] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:51.659405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.668985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee084a1-e6f2-49b4-a72f-107db9c48c85 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.673624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.674513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.674796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.675074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] PATH_INFO: `/v3/projects/0e710d572e694676b860169c8731c2f3/users/1ad4f1c87a9644a7a21cb27dbc76500a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.690671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e710d572e694676b860169c8731c2f3, user_id=1ad4f1c87a9644a7a21cb27dbc76500a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.694034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:51.738380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1151/2286] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:51 2022] PUT /identity/v3/projects/0e710d572e694676b860169c8731c2f3/users/1ad4f1c87a9644a7a21cb27dbc76500a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:51.748993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.749463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.749840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.776105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] MFA Rules not processed for user `1ad4f1c87a9644a7a21cb27dbc76500a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:51.777308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:51.839298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1136/2287] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:51.843665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:51.917466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e84e-4597-47b3-8a01-241b0156fd43 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:51.922520 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:51.923415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:51.923795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:51.924172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:51.924998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:51.927651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.038378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:52.053205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1152/2288] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:28:51 2022] POST /identity/v3/projects => generated 363 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:28:52.057183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.068146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494a8e-34eb-4c2e-85c7-b5bc8dcbf6cb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.126960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:52.133677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.134419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.134687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.134965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.135683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.137632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.220746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1137/2289] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:28:52 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:28:52.230078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.240666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9fbd1-ce3f-492c-bd46-cfaa9ea457a1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.245279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.245995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.246315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.246588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.247313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.248901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.260045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1153/2290] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:52.264313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.274965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fde401-d9b8-4396-a16f-f0f30483a9fc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.279770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.280682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.280973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.281250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] PATH_INFO: `/v3/projects/a1b373946e594c73ad6d50ac36e8c025/users/2bb29d5d3ae24286a7c27d81638349e3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.297111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b373946e594c73ad6d50ac36e8c025, user_id=2bb29d5d3ae24286a7c27d81638349e3, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.300797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.409565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1138/2291] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:52 2022] PUT /identity/v3/projects/a1b373946e594c73ad6d50ac36e8c025/users/2bb29d5d3ae24286a7c27d81638349e3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:52.414322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.424202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df2741-3635-4066-8995-223429d5ee5c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.480691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.481422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.481687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.481958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.482719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.484390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c089d-3239-4cf1-a48f-86a0a0231b2a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.494610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1154/2292] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:28:52.498939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.511875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537949-1dfc-4b38-9475-9a07cf43b51f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.512792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.513065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.513354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.517263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.518283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.518714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.518978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] PATH_INFO: `/v3/projects/a1b373946e594c73ad6d50ac36e8c025/users/2bb29d5d3ae24286a7c27d81638349e3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.521457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b373946e594c73ad6d50ac36e8c025, user_id=2bb29d5d3ae24286a7c27d81638349e3, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.527867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735e13d-82b1-45a7-b03a-3e570677a55f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.537746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1139/2293] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:28:52 2022] PUT /identity/v3/projects/a1b373946e594c73ad6d50ac36e8c025/users/2bb29d5d3ae24286a7c27d81638349e3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:52.542492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:52.543898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:52.548784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.549076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.549371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.585227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] MFA Rules not processed for user `2bb29d5d3ae24286a7c27d81638349e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:52.586587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:52.628721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1155/2294] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:28:52.664075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1140/2295] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:28:52.675289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.677069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:52.733559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e267d4-310d-44b0-9a46-a493d84d2211 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.737199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.737839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.738149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.738444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.747317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.748778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.758640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1156/2296] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:52.766465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.768070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:52.839956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8cdb-c7a6-430c-8c0a-5180ea3b4c83 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.845330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.845895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.846208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.846484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.855396 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:52.857078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:52.865339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1141/2297] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:28:52.915996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:52.927782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36ba72-3136-43aa-ae70-6c928ace3840 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:52.931653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:52.932306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:52.932645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:52.932933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:52.968175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:53.004387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:53.006567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:53.022093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1157/2298] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:53.524720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:53.526599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:53.560885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:53.638569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4475-8dcf-4053-9299-ad38d04b7a18 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:53.646467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:53.647369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:53.647656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:53.647937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:53.651626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:53.717909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:53.719962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90036f86-82cc-4684-850a-da64ca9ea6fe service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:53.734434 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1142/2299] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:28:56.868312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:56.944841 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01bfc4-331a-4e58-934c-c5c4c3e7445c service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:56.951842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:56.952937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:56.953422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:56.954990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] PATH_INFO: `/v3/users/1ad4f1c87a9644a7a21cb27dbc76500a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:56.956401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad4f1c87a9644a7a21cb27dbc76500a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:56.959236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:57.150787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad4f1c87a9644a7a21cb27dbc76500a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:57.151590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad4f1c87a9644a7a21cb27dbc76500a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:57.152202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad4f1c87a9644a7a21cb27dbc76500a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:57.166668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad4f1c87a9644a7a21cb27dbc76500a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:57.173607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad4f1c87a9644a7a21cb27dbc76500a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:57.237642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1158/2300] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:28:56 2022] DELETE /identity/v3/users/1ad4f1c87a9644a7a21cb27dbc76500a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:57.241907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:57.242408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:57.242653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:57.263723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:57.264915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:57.330846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1143/2301] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:57.341647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:57.343084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:57.424679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34d8c-e447-4db1-8c20-dea6714e694f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:57.429388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:57.430069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:57.430372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:57.430649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:57.437349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:57.441759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:57.443406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:57.454956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1159/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:28:57.960627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:57.961983 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:57.980054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc448-1391-49a4-9b5a-87aa815aa485 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:57.990816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:58.003936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:58.004488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:58.004910 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] PATH_INFO: `/v3/projects/0e710d572e694676b860169c8731c2f3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:58.009162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e710d572e694676b860169c8731c2f3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:58.012213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:58.141256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e710d572e694676b860169c8731c2f3', 'request_id': 'req-bea80cda-cfad-4418-b88d-170add52b820'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.146822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:58.253532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e710d572e694676b860169c8731c2f3', 'request_id': 'req-bea80cda-cfad-4418-b88d-170add52b820'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.261519 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e710d572e694676b860169c8731c2f3', 'request_id': 'req-bea80cda-cfad-4418-b88d-170add52b820'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.310932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] The token cache is being invalidate because project 0e710d572e694676b860169c8731c2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:28:58.313526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.313526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.313875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1144/2303] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:28:57 2022] DELETE /identity/v3/projects/0e710d572e694676b860169c8731c2f3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:58.318253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:58.330450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28a34-2a6a-48f7-8120-244e02d32494 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:58.393364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:58.394434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:58.395306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:58.395682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] PATH_INFO: `/v3/users/2bb29d5d3ae24286a7c27d81638349e3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:58.397109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb29d5d3ae24286a7c27d81638349e3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:58.399556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:58.577862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb29d5d3ae24286a7c27d81638349e3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.578628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb29d5d3ae24286a7c27d81638349e3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.579242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb29d5d3ae24286a7c27d81638349e3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.593150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb29d5d3ae24286a7c27d81638349e3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.601667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7219e8ff-7117-4979-9934-7aeb9da46e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb29d5d3ae24286a7c27d81638349e3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:58.691328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1160/2304] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:28:58 2022] DELETE /identity/v3/users/2bb29d5d3ae24286a7c27d81638349e3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:59.180749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:59.182272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:59.199438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80cda-cfad-4418-b88d-170add52b820 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:59.281806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:59.281806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:59.281806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:59.281806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] PATH_INFO: `/v3/projects/a1b373946e594c73ad6d50ac36e8c025` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:59.282911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b373946e594c73ad6d50ac36e8c025)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:59.282911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:59.613240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b373946e594c73ad6d50ac36e8c025', 'request_id': 'req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:59.727739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b373946e594c73ad6d50ac36e8c025', 'request_id': 'req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:59.735198 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b373946e594c73ad6d50ac36e8c025', 'request_id': 'req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:59.750747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] The token cache is being invalidate because project a1b373946e594c73ad6d50ac36e8c025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:28:59.751048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:59.751706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6f511a2-a623-4038-9c30-8a41b6ea9eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:28:59.753255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1145/2305] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:28:59 2022] DELETE /identity/v3/projects/a1b373946e594c73ad6d50ac36e8c025 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:28:59.758803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:59.759659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:59.760946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:59.781789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:59.783033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:59.862388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1161/2306] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:59.870362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:59.870655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:59.871072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:59.891914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:28:59.893415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:28:59.908161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1146/2307] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:28:59.912435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:59.925219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517ab38-1fdf-453c-81fd-6fea0620e76d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:59.930580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:59.931246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:59.931542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:59.931813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:59.932545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:28:59.932852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:59.934890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:28:59.945349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:28:59.950072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1162/2308] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:28:59.958133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:28:59.965996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b71e2c-7587-47e3-8edd-04bd0b30522d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:28:59.971511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:28:59.972606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:28:59.973082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:28:59.973920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:28:59.974661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:28:59.976605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.061846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1147/2309] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:28:59 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:00.066716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.080601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448435b4-4463-4b15-a799-7aa982191430 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:00.171309 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:00.172140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.172438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.172778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.173552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:00.175839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.272344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1163/2310] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:00 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:00.276332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.292680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de4a4-80eb-4c07-89bd-9089e391e481 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:00.298980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:00.299722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.300009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.300280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.301059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:00.303531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.315586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1148/2311] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:00.319660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.332786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089be3-616a-4c06-9c05-ea8942a715ba admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:00.339531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:00.340311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.340721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.341003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.341747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:00.343428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb019-1fe6-4c3b-9dcb-db15887625cb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.359441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1164/2312] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:00.363009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.375988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22666419-0399-4950-b9da-65f94990c182 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:00.382773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:00.382773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.382773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.383192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] PATH_INFO: `/v3/projects/74e8eee150784dd891d4a683246cd402/users/b66f0a96e64e4da2890984b26a2194e1/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.401027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e8eee150784dd891d4a683246cd402, user_id=b66f0a96e64e4da2890984b26a2194e1, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:00.403678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.477963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1149/2313] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:00 2022] PUT /identity/v3/projects/74e8eee150784dd891d4a683246cd402/users/b66f0a96e64e4da2890984b26a2194e1/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:00.493172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.493495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.493729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.541604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] MFA Rules not processed for user `b66f0a96e64e4da2890984b26a2194e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:00.544322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:00.677527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1165/2314] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:00.677757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.696470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:00.783695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b87e2-cba6-4138-b1e5-c93b4ac519bb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:00.794176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:00.794860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:00.795142 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:00.795427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:00.796181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:00.798144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:00.917578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:00.929928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1150/2315] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:00 2022] POST /identity/v3/projects => generated 389 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:00.934918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27adb5-9b8c-4dcc-adf5-2cce53746784 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.010529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.011372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.011777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.012110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.012913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.015828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.122907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:01.135591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:01.141384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1166/2316] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:00 2022] POST /identity/v3/users => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:01.147710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9262bd3-8ea9-4818-a982-f76bf6edf27b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.155640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.156284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.157484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.157698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.158475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.161310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.183689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1151/2317] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:01.193230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:01.206313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969bff9-9ede-4205-8089-c226e139892d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.220981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.222071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.224236 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.225546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] PATH_INFO: `/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f/users/8647f47727b944b9964fd81bb5c09979/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.264364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad367ce0ca2e48a1974a17b9c8de474f, user_id=8647f47727b944b9964fd81bb5c09979, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.271510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.332441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:01.339311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1167/2318] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:01 2022] PUT /identity/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f/users/8647f47727b944b9964fd81bb5c09979/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:01.350224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afbcaa-d3bc-4a1a-9237-6c230b684215 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.442735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.443762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.444072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.444449 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.445809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.466482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9670a-0ccd-476b-9aae-7291a24b54a5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.495099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1152/2319] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:01.499480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:01.517234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1268d-31b5-4343-8424-a916a456071b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.541196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.544144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.544584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.546446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] PATH_INFO: `/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f/users/8647f47727b944b9964fd81bb5c09979/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.551058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad367ce0ca2e48a1974a17b9c8de474f, user_id=8647f47727b944b9964fd81bb5c09979, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.557442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.586374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1168/2320] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:01 2022] PUT /identity/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f/users/8647f47727b944b9964fd81bb5c09979/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:01.597153 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.597529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.597912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.667921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] MFA Rules not processed for user `8647f47727b944b9964fd81bb5c09979`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:01.669754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:01.794588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1153/2321] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:01.800797 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:01.803103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:01.906705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44c7c-2956-4cd1-a4bb-fe1d66cd6747 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:01.925438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:01.926037 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:01.926926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:01.927788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:01.947587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:01.949499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba055487-dcea-4d07-b2ab-8e0ddf9a59ba service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:01.974825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1169/2322] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:02.159912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:02.166179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:02.260166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cea03d-89f6-4bd3-9d4d-ad61e949316a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:02.274795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:02.275742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:02.276023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:02.282282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] PATH_INFO: `/v3/users/b66f0a96e64e4da2890984b26a2194e1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:02.286287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66f0a96e64e4da2890984b26a2194e1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:02.287119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:02.488425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66f0a96e64e4da2890984b26a2194e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:02.489193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66f0a96e64e4da2890984b26a2194e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:02.489772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66f0a96e64e4da2890984b26a2194e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:02.507557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66f0a96e64e4da2890984b26a2194e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:02.524736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66f0a96e64e4da2890984b26a2194e1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:02.636698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1154/2323] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:02 2022] DELETE /identity/v3/users/b66f0a96e64e4da2890984b26a2194e1 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:02.638546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:02.639046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:02.639509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:02.669565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:02.671214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:02.744577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1170/2324] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:02.755038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:02.756809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:02.829784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b714-fbc3-452d-ae20-1063fab594b9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:02.834135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:02.834818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:02.835133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:02.835447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:02.849998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:02.851534 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:02.866033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1155/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:03.592287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:03.594347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:03.610696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280485-a5f3-4aed-8a40-2de8dc73f750 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:03.615433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:03.616301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:03.616583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:03.616894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] PATH_INFO: `/v3/projects/74e8eee150784dd891d4a683246cd402` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:03.618203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e8eee150784dd891d4a683246cd402)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:03.620143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:03.754674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e8eee150784dd891d4a683246cd402', 'request_id': 'req-8c6f7367-af4d-4795-86ac-9754c0ac6968'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:03.819367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e8eee150784dd891d4a683246cd402', 'request_id': 'req-8c6f7367-af4d-4795-86ac-9754c0ac6968'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:03.821747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:03.827407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e8eee150784dd891d4a683246cd402', 'request_id': 'req-8c6f7367-af4d-4795-86ac-9754c0ac6968'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:03.842664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] The token cache is being invalidate because project 74e8eee150784dd891d4a683246cd402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:03.843049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:03.844416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:03.850784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1171/2326] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:03 2022] DELETE /identity/v3/projects/74e8eee150784dd891d4a683246cd402 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:03.857966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:03.861373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fab220-380b-45eb-80c3-e4079f51ca00 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:03.901249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:03.944237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:03.945664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:03.946106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:03.946507 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] PATH_INFO: `/v3/users/8647f47727b944b9964fd81bb5c09979` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:03.948261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8647f47727b944b9964fd81bb5c09979)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:03.950695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:04.149111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8647f47727b944b9964fd81bb5c09979'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:04.150051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8647f47727b944b9964fd81bb5c09979'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:04.150747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8647f47727b944b9964fd81bb5c09979'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:04.167861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8647f47727b944b9964fd81bb5c09979'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:04.175796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-956f9218-3a01-4e0b-9cb7-a9f5bab5556c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8647f47727b944b9964fd81bb5c09979'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:04.305304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1156/2327] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:03 2022] DELETE /identity/v3/users/8647f47727b944b9964fd81bb5c09979 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:04.841333 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:04.843246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:04.861676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7367-af4d-4795-86ac-9754c0ac6968 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:04.921172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:04.922042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:04.922334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:04.922622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] PATH_INFO: `/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:04.923769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad367ce0ca2e48a1974a17b9c8de474f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:04.925657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.018516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad367ce0ca2e48a1974a17b9c8de474f', 'request_id': 'req-38998c83-d1af-41a4-9752-96e6c54a99b6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:05.132888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad367ce0ca2e48a1974a17b9c8de474f', 'request_id': 'req-38998c83-d1af-41a4-9752-96e6c54a99b6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:05.139747 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad367ce0ca2e48a1974a17b9c8de474f', 'request_id': 'req-38998c83-d1af-41a4-9752-96e6c54a99b6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:05.150398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] The token cache is being invalidate because project ad367ce0ca2e48a1974a17b9c8de474f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:05.150671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:05.151284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-38998c83-d1af-41a4-9752-96e6c54a99b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:05.152793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1172/2328] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:04 2022] DELETE /identity/v3/projects/ad367ce0ca2e48a1974a17b9c8de474f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:05.157780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.158264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.158376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.176930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:05.178201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:05.242883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1157/2329] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:05.251460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.251830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.252162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.271005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:05.272241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:05.280252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1173/2330] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:05.284601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.295250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897195d-7215-4cf6-a26d-4144a01e5f68 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.299894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.303260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.303260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.303260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.303260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:05.303260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.303698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.314840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1158/2331] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:29:05.319538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.334325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51579-d135-47cf-a53b-6a8c3cf4b700 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.339324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.340025 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.340370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.340650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.341352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.344495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.432840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1174/2332] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:05.437185 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.447989 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530f623-24dc-446c-95b6-64e55b086fff admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.488370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:05.518573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.519251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.519537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.519818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.521146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.523084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.625210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1159/2333] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:05.628890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.639562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7d4a1-a54a-47d4-985e-ee0aa709f4a9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.644125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.644787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.645723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.645995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.646753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.648355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.659163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1175/2334] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:05.663085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.673156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac111ba-f70a-4e22-b4cc-f3b5af80e862 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.678435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.680592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.680592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.680592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.680592 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.682341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67a16f-a52e-452e-87fa-c699b6214ba3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.693072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1160/2335] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:05.697300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.707743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77486fc5-fadb-4735-bdc6-692d2a63d776 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.712450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.713331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.713616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.713895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] PATH_INFO: `/v3/projects/73aece33655145118acda23fd4cbe5c5/users/67cf7c839366491a938db5735ee888b4/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.729089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aece33655145118acda23fd4cbe5c5, user_id=67cf7c839366491a938db5735ee888b4, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.731854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:05.801996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1176/2336] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:05 2022] PUT /identity/v3/projects/73aece33655145118acda23fd4cbe5c5/users/67cf7c839366491a938db5735ee888b4/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:05.812967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.813255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.813557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.839787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] MFA Rules not processed for user `67cf7c839366491a938db5735ee888b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:05.840974 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:05.908782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1161/2337] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:05.913224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:05.926913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:05.974110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1422a2-12ec-4264-a325-922beae6e3e6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:05.978864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:05.979756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:05.980039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:05.980318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:05.981011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:05.982999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.084232 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.084429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1177/2338] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:05 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:06.107179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf8308-e13d-46b1-b4f4-ff41e1a9eeb4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.171593 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.172291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.172574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.172850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.173554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.175536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.260640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1162/2339] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:06 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:06.265012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.276281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995dc9-7004-4fa0-9f86-575bfaf2a1fe admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.281070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.281757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.282151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.282430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.283330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.285010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.296654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1178/2340] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:06.300931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.312000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4e188-0791-40d6-aa7d-d610e7aad6cf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.316614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.317584 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.317874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.318187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] PATH_INFO: `/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c/users/19944839ebe547a8920b1a5326768d14/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.333830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ed3e5c7dfe477181a98152a38d5a4c, user_id=19944839ebe547a8920b1a5326768d14, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.336609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.399859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1163/2341] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:06 2022] PUT /identity/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c/users/19944839ebe547a8920b1a5326768d14/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:06.403518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.414319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcd19e-43de-4c7c-80a5-31360d3a1e2d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.470135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.470812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.471093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.471377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.472104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.474424 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3ab23-3e10-43ac-ae03-622c9043f8ff admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.484441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1179/2342] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:06.488590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.499278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd640e-f8fc-4472-8f21-2b9329433eab admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.503960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.504920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.505200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.505473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] PATH_INFO: `/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c/users/19944839ebe547a8920b1a5326768d14/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.507478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ed3e5c7dfe477181a98152a38d5a4c, user_id=19944839ebe547a8920b1a5326768d14, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.510186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.512532 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:06.521313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1164/2343] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:06 2022] PUT /identity/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c/users/19944839ebe547a8920b1a5326768d14/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:06.531492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.531779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.532058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.558009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] MFA Rules not processed for user `19944839ebe547a8920b1a5326768d14`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:06.559237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:06.629299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1180/2344] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:06.639939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:06.642226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:06.698972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c8f0-61f0-413e-9bc8-c479cb804cb5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:06.702585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:06.703267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:06.703547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:06.703849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:06.713555 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:06.715060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d684c-0af2-4587-b0fe-5cd4f4160b3f service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:06.725390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1165/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:07.202191 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:07.211110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:07.270566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123cf08-b465-4411-adc8-b906af0f6bcd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:07.275233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:07.276097 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:07.276380 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:07.276655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] PATH_INFO: `/v3/users/67cf7c839366491a938db5735ee888b4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:07.277827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67cf7c839366491a938db5735ee888b4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:07.279891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:07.403268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67cf7c839366491a938db5735ee888b4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:07.404015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67cf7c839366491a938db5735ee888b4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:07.404772 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67cf7c839366491a938db5735ee888b4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:07.416837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67cf7c839366491a938db5735ee888b4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:07.423658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67cf7c839366491a938db5735ee888b4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:07.481072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1181/2346] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:07 2022] DELETE /identity/v3/users/67cf7c839366491a938db5735ee888b4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:07.490374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:07.490837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:07.490933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:07.508926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:07.510138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:07.566023 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1166/2347] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:07.576060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:07.577473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:07.634775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946872c-456f-4093-86d6-851a9bd93bb5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:07.638431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:07.639080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:07.639358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:07.639657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:07.648813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:07.650302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:07.659756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1182/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:08.096943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:08.099109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:08.111731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172b14-ad51-4875-9efe-8c98f173abc4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:08.121179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:08.122181 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:08.122364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:08.122666 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] PATH_INFO: `/v3/projects/73aece33655145118acda23fd4cbe5c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:08.123829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73aece33655145118acda23fd4cbe5c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:08.125748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:08.233977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73aece33655145118acda23fd4cbe5c5', 'request_id': 'req-db556e79-c38b-4524-bef5-696028348765'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.293348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73aece33655145118acda23fd4cbe5c5', 'request_id': 'req-db556e79-c38b-4524-bef5-696028348765'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.300686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73aece33655145118acda23fd4cbe5c5', 'request_id': 'req-db556e79-c38b-4524-bef5-696028348765'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.320246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] The token cache is being invalidate because project 73aece33655145118acda23fd4cbe5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:08.320708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.321366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.323467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1167/2349] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:08 2022] DELETE /identity/v3/projects/73aece33655145118acda23fd4cbe5c5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:08.327487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:08.340660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85a12-0059-4bc6-b788-07ec2cf51c61 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:08.374986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:08.428948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:08.430665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:08.431314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:08.431314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] PATH_INFO: `/v3/users/19944839ebe547a8920b1a5326768d14` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:08.432506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19944839ebe547a8920b1a5326768d14)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:08.434803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:08.505102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:08.506586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:08.530730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:08.567129 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556e79-c38b-4524-bef5-696028348765 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:08.570813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:08.571469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:08.571788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:08.572055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:08.574127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:08.587980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:08.589837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:08.600210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1168/2350] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:08.626783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19944839ebe547a8920b1a5326768d14'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.627499 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19944839ebe547a8920b1a5326768d14'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.628160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19944839ebe547a8920b1a5326768d14'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.640904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19944839ebe547a8920b1a5326768d14'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.648343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b343d1e8-2d21-42d1-a8e9-37e88e71f7e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19944839ebe547a8920b1a5326768d14'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:08.756625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1183/2351] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:08 2022] DELETE /identity/v3/users/19944839ebe547a8920b1a5326768d14 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:09.249454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:09.251804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:09.265551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11fded-0c8d-423d-a67f-8b55122c55c5 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:09.335149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:09.336101 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.336453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.336871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] PATH_INFO: `/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.338420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ed3e5c7dfe477181a98152a38d5a4c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:09.340445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:09.448223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ed3e5c7dfe477181a98152a38d5a4c', 'request_id': 'req-0b93cf97-b9ae-4127-bc25-d8d5fb943772'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:09.532911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ed3e5c7dfe477181a98152a38d5a4c', 'request_id': 'req-0b93cf97-b9ae-4127-bc25-d8d5fb943772'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:09.539929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ed3e5c7dfe477181a98152a38d5a4c', 'request_id': 'req-0b93cf97-b9ae-4127-bc25-d8d5fb943772'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:09.552748 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] The token cache is being invalidate because project f5ed3e5c7dfe477181a98152a38d5a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:09.553034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:09.553745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0b93cf97-b9ae-4127-bc25-d8d5fb943772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:09.555317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1169/2352] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:09 2022] DELETE /identity/v3/projects/f5ed3e5c7dfe477181a98152a38d5a4c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:09.560805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.561286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.561383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.580637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:09.581833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:09.641813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1184/2353] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:09.651229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.651606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.651892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.671388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:09.672626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:09.680622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1170/2354] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:09.684895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:09.694768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7621212-98cf-4769-a7b2-6476b46ab3fb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:09.700277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:09.700459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.700670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.700953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.701722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:09.702078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:09.703670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:09.715709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1185/2355] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:29:09.726103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:09.735481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29eb0d9-a21a-451c-a7c9-4d3400147493 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:09.740481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:09.741130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.741413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.741687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.742470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:09.744369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:09.847557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1171/2356] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:09 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:09.851634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:09.865140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aceaa9-c243-4673-98f4-fd13204806a2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:09.907653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:09.935983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:09.936674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:09.936962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:09.937374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:09.938130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:09.940288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.021733 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1186/2357] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:09 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:10.029164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.042057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f605c-423c-48d1-b718-43eca2f4d363 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.047197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.049107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.049107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.049107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.049509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.051283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.072980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1172/2358] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:10.073822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.086119 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b08130-58b0-4d78-946f-11ad8ff4e7d4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.090761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.091613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.091996 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.092373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.093199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.094962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adc4d5-d436-4e56-8392-981d45b39543 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.105657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1187/2359] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:10.109968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.123205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962cac0-930e-4b9c-afc1-2e3f3d4b9f43 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.128787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.130253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.130536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.130813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] PATH_INFO: `/v3/projects/8c619f909af24844a25c70019c317552/users/4c7c4b88b28b43ca8be115db113d5e87/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.143804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:10.148443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c619f909af24844a25c70019c317552, user_id=4c7c4b88b28b43ca8be115db113d5e87, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.152816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.239155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1173/2360] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:10 2022] PUT /identity/v3/projects/8c619f909af24844a25c70019c317552/users/4c7c4b88b28b43ca8be115db113d5e87/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:10.256046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.256692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.257113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.284413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] MFA Rules not processed for user `4c7c4b88b28b43ca8be115db113d5e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:10.285703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:10.359813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1188/2361] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:10.365301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.445073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c46ee-22fb-40c1-b8c2-8b5c80b8ca1b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.451577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.452330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.452677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.453018 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.453811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.456133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.531060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1174/2362] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:10 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:10.534785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.554264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd7843-ca0c-4c7b-982a-6fea8565fdc0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.633080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.633080 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.633773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.633773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.634449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.636687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.721192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1189/2363] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:10 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:10.723462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.737617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3615f6-c2b0-40b9-ba51-fdd644dc53c0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.743566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.744419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.744715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.745000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.745927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.747686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.764123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1175/2364] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:10.766130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.779469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc407d-7616-4f37-b2a0-09f66afef847 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.786030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.787061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.787629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.787923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] PATH_INFO: `/v3/projects/83f0fe60c743415d820b05c9e352899c/users/806e790ca9e0494fb93ea21b988d5ed9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.807257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f0fe60c743415d820b05c9e352899c, user_id=806e790ca9e0494fb93ea21b988d5ed9, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.810522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.896079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1190/2365] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:10 2022] PUT /identity/v3/projects/83f0fe60c743415d820b05c9e352899c/users/806e790ca9e0494fb93ea21b988d5ed9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:10.897583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:10.907987 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1476e-7967-4e14-bbcb-e8139d2c5e1b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:10.981021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:10.981789 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:10.982121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:10.982406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:10.983442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:10.985026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef8196-cc84-4db5-80bb-1e7e64df8f8b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:10.996966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1176/2366] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:11.001142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:11.015063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958baa-fc24-4bd5-bb8c-b5e1d61bab05 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:11.022744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:11.023813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.024200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.024607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] PATH_INFO: `/v3/projects/83f0fe60c743415d820b05c9e352899c/users/806e790ca9e0494fb93ea21b988d5ed9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.027383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f0fe60c743415d820b05c9e352899c, user_id=806e790ca9e0494fb93ea21b988d5ed9, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:11.030621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:11.039668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:11.054064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1191/2367] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:10 2022] PUT /identity/v3/projects/83f0fe60c743415d820b05c9e352899c/users/806e790ca9e0494fb93ea21b988d5ed9/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:11.061420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.061839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.062252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.092446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] MFA Rules not processed for user `806e790ca9e0494fb93ea21b988d5ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:11.096866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:11.162963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:11.211089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1177/2368] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:11.216615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:11.224647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:11.292163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4386de-fed0-435d-98c5-1d733e1ef086 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:11.296519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:11.297237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.297670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.298204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.313086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:11.317902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbb4c3-1d41-417b-97b0-8d97fe95d99a service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:11.336995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1192/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:11.545059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:11.554738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83137a-2cc6-4de1-bec3-5c7b37172a48 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:11.558335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:11.567480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.567480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.567480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.642743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:11.649194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f30ddb-7897-4753-88c5-3546661d2d86 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:11.677066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1178/2370] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:11.702280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.705561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.705561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.705561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1193/2371] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:29:11.714770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:11.714770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:11.714770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:11.761315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:11.762796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:11.907682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1179/2372] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:11.920891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:11.929885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.081691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f1fe-425e-4842-821d-ae588b661906 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:12.086130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:12.086910 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.087538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.087911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.107695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:12.114190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2884e-df8e-4c08-88eb-bd043d20ab23 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:12.130042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1194/2373] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:12.338788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:12.346651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fb47-db28-455f-83f9-e13116c1b15e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.346953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fb47-db28-455f-83f9-e13116c1b15e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.347273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fb47-db28-455f-83f9-e13116c1b15e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.349967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1195/2374] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 20:29:12.356255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.356692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.356919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.392837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] MFA Rules not processed for user `617e5a9c04ec4d5fa7833ab46264142d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:12.395049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.509056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14b881-3ce6-47c4-b292-6ae075f322ba None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:12.519100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:12.522443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.522443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.522443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] PATH_INFO: `/v3/users/4c7c4b88b28b43ca8be115db113d5e87` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.523671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7c4b88b28b43ca8be115db113d5e87)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:12.526754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:12.528486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1196/2375] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:12.541885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:12.543479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.559887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06312fd-bce9-46ad-9144-c43f94c580bd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:12.564261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:12.565325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.565923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.566498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.568964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:12.578185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:12.581379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabfc6f-6849-45ad-b844-e85a9183013b service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:12.590954 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1197/2376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:12.709746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759e74-6086-4037-aadf-4d89a0708ed9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.710637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759e74-6086-4037-aadf-4d89a0708ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.710637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759e74-6086-4037-aadf-4d89a0708ed9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.712817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1198/2377] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 20:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 20:29:12.718726 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.719083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.719384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.738782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] MFA Rules not processed for user `4cf871584be84949a82a9da23fb16153`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:12.740015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.750230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1199/2378] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:12.761361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:12.762715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.786340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d7dc3-4462-4198-8b21-c88289dfa34a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:12.789793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7c4b88b28b43ca8be115db113d5e87'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:12.790511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7c4b88b28b43ca8be115db113d5e87'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:12.791127 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7c4b88b28b43ca8be115db113d5e87'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:12.793470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:12.794496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.794888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.795269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.797669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.816140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7c4b88b28b43ca8be115db113d5e87'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:12.828503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:12.831521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7c4b88b28b43ca8be115db113d5e87'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:12.833361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23b6ca-c970-4a62-ac84-fae3f33e5990 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:12.847684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1200/2379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:12.929212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1180/2380] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:12 2022] DELETE /identity/v3/users/4c7c4b88b28b43ca8be115db113d5e87 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:12.935805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:12.936198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:12.936609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:12.970819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:12.972976 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:12.988426 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:13.070723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:13.072345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:13.074884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1201/2381] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:13.181489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99887-58a1-4c23-a7cf-c6e74fe2fdbb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:13.187825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:13.191005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:13.192010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:13.192479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:13.207637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:13.210705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:13.228801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1181/2382] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:13.826562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:13.828814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:13.842969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01594-0ed9-4311-9b44-9fbc036eed37 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:13.858136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:13.858136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:13.858136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:13.858136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] PATH_INFO: `/v3/users/0a19c51d287046578e9e4016784dad8f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:13.859079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a19c51d287046578e9e4016784dad8f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:13.859079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:13.859936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:13.861360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:13.881727 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98077e76-7a01-47c0-a4f3-e41882771509 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:13.888515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:13.889430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:13.889878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:13.890454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] PATH_INFO: `/v3/projects/8c619f909af24844a25c70019c317552` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:13.895269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c619f909af24844a25c70019c317552)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:13.897213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:14.078780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a19c51d287046578e9e4016784dad8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.078998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c619f909af24844a25c70019c317552', 'request_id': 'req-b6551674-678e-4231-b9a1-41641343709f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.079619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a19c51d287046578e9e4016784dad8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.080267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a19c51d287046578e9e4016784dad8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.098508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a19c51d287046578e9e4016784dad8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.107774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a19c51d287046578e9e4016784dad8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.143494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c619f909af24844a25c70019c317552', 'request_id': 'req-b6551674-678e-4231-b9a1-41641343709f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.147075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c619f909af24844a25c70019c317552', 'request_id': 'req-b6551674-678e-4231-b9a1-41641343709f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.159061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] The token cache is being invalidate because project 8c619f909af24844a25c70019c317552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:14.159356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.160037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.162797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1182/2383] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:13 2022] DELETE /identity/v3/projects/8c619f909af24844a25c70019c317552 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:14.166637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:14.167977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:14.191112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:14.242270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6551674-678e-4231-b9a1-41641343709f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:14.247212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:14.248098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:14.248375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:14.248646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] PATH_INFO: `/v3/users/806e790ca9e0494fb93ea21b988d5ed9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:14.249848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806e790ca9e0494fb93ea21b988d5ed9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:14.251846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:14.265030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1202/2384] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:13 2022] DELETE /identity/v3/users/0a19c51d287046578e9e4016784dad8f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:14.403545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806e790ca9e0494fb93ea21b988d5ed9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.404830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806e790ca9e0494fb93ea21b988d5ed9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.406254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806e790ca9e0494fb93ea21b988d5ed9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.429249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806e790ca9e0494fb93ea21b988d5ed9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.436312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-966516c5-0b16-4539-ac62-ac6d5ac75f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806e790ca9e0494fb93ea21b988d5ed9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.485442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:14.486899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:14.514286 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1183/2385] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:14 2022] DELETE /identity/v3/users/806e790ca9e0494fb93ea21b988d5ed9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:14.578681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14aa4-5fa1-4b02-a1d6-d4360332b435 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:14.623817 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:14.669015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:14.670374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:14.670782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:14.671122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] PATH_INFO: `/v3/projects/39849cec37754352a6822cd57277b954` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:14.672569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39849cec37754352a6822cd57277b954)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:14.674722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:14.815746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39849cec37754352a6822cd57277b954', 'request_id': 'req-fdc681db-9aae-44c8-8bb9-982b88eff513'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.869845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39849cec37754352a6822cd57277b954', 'request_id': 'req-fdc681db-9aae-44c8-8bb9-982b88eff513'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.887994 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39849cec37754352a6822cd57277b954', 'request_id': 'req-fdc681db-9aae-44c8-8bb9-982b88eff513'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.899923 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] The token cache is being invalidate because project 39849cec37754352a6822cd57277b954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:14.900137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.901259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fdc681db-9aae-44c8-8bb9-982b88eff513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:14.903149 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1203/2386] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:14 2022] DELETE /identity/v3/projects/39849cec37754352a6822cd57277b954 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:14.908289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:14.908950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:14.908950 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:14.930311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:14.931885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.008948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1184/2387] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:15.017312 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.017608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.017887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.039184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:15.040402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.050444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1204/2388] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:15.055295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.067670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafe6fb-e427-4622-9110-70f74feb0b4b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.070872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.072632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.074596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.075333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.075614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.075889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.076625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:15.076953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.078691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.092350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cb26e-fa4c-43b0-85e4-0f2b09741a1d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.098263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.099461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.100214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.100498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] PATH_INFO: `/v3/projects/83f0fe60c743415d820b05c9e352899c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.101813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f0fe60c743415d820b05c9e352899c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.103949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.105070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1185/2389] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:29:15.109107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.120344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5093eba-5c98-41fe-827e-eff4f9b7e6ab admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.125362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.126079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.126363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.126636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.127345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.132254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.233649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1186/2390] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:15.237772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.244882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:15.248738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f0fe60c743415d820b05c9e352899c', 'request_id': 'req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:15.303863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef0896-cd64-4be2-a1ab-54fcec18ca9a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.308957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.309625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.309906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.310227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.311290 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.317289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.333144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f0fe60c743415d820b05c9e352899c', 'request_id': 'req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:15.340099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f0fe60c743415d820b05c9e352899c', 'request_id': 'req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:15.351973 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] The token cache is being invalidate because project 83f0fe60c743415d820b05c9e352899c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:15.352261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:15.352944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7735c086-0a9c-4ed1-b8bb-522119f3f5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:15.354584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1205/2391] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:15 2022] DELETE /identity/v3/projects/83f0fe60c743415d820b05c9e352899c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:15.359227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f433-e66e-4a36-8af1-19384cb21176 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.359517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f433-e66e-4a36-8af1-19384cb21176 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.359798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f433-e66e-4a36-8af1-19384cb21176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.377918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9ce3f433-e66e-4a36-8af1-19384cb21176 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:15.379166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9ce3f433-e66e-4a36-8af1-19384cb21176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.387940 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1187/2392] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/users => generated 303 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:15.388085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1206/2393] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:15.389179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.390885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.396768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.397094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.397318 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.404882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48796e94-0a18-4621-bcd0-98c9a704b725 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.410819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.411421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.411858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.412437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.413170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.415301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.421608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:15.433523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.433680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1188/2394] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:15.433795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1207/2395] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:15.433889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.435143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.441956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e64408-655a-48d6-993e-05236cd13d02 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.444065 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fdaa1-82a6-48e4-b88e-7e788e17b859 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.446956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.447613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.447897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.448174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.449132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.449298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.449787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.450107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.450385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.450770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.451114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:15.451422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.452998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.464002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1189/2396] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:15.470558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1208/2397] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:29:15.471373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.471684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.478560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93e5e-feee-427e-a46c-1ecc03987dd7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.481912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89148-d214-41d0-9ccb-da5cfa89325b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.484045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.484970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.485258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.485537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] PATH_INFO: `/v3/projects/b0d06240acd94a6ebf3d9b4b1048a6b8/users/a256c5b8d1464727ab4e31539c999ec0/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.486533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.487170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.487448 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.487720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.488454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.490495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.501562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d06240acd94a6ebf3d9b4b1048a6b8, user_id=a256c5b8d1464727ab4e31539c999ec0, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.504563 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313362f-5084-4c0f-9978-e4d2acc04504 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.607472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:15.613073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1190/2398] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:15 2022] PUT /identity/v3/projects/b0d06240acd94a6ebf3d9b4b1048a6b8/users/a256c5b8d1464727ab4e31539c999ec0/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:15.614400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacfcf2-f574-4f2c-a4e0-b43b683cf693 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.614691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacfcf2-f574-4f2c-a4e0-b43b683cf693 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.614801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1209/2399] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/projects => generated 397 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:15.614969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacfcf2-f574-4f2c-a4e0-b43b683cf693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.619109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.633375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2eacfcf2-f574-4f2c-a4e0-b43b683cf693 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:15.634790 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2eacfcf2-f574-4f2c-a4e0-b43b683cf693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.680614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6b19-a492-4267-8ddd-e9ba13500bbe admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.685631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.686222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.686505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.686779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.687477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.689592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.689975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1191/2400] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:15.691452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.691607 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.691884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.718197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] MFA Rules not processed for user `a256c5b8d1464727ab4e31539c999ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:15.719486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.785892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1192/2401] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:15.785892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.785892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:15.790411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1210/2402] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:15.790411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.792812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:15.797735 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb043409-3773-4221-add4-c1efe1c49ac5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.802896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.803537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.803816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.804086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.804769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.806701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.855886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f33d85-e487-41ed-b5bd-1c3fba688572 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.861154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e283df-96fc-4711-9782-13e33899433a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.862102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e283df-96fc-4711-9782-13e33899433a service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.862403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e283df-96fc-4711-9782-13e33899433a service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.862712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e283df-96fc-4711-9782-13e33899433a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.873055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e283df-96fc-4711-9782-13e33899433a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.875059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e283df-96fc-4711-9782-13e33899433a service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.878756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1211/2403] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:15.878756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.885568 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1193/2404] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:29:15.897433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e283df-96fc-4711-9782-13e33899433a service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:15.906197 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e283df-96fc-4711-9782-13e33899433a service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.980734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.982110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.982581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.983051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.983751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.984219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8329-b0b8-4e87-b380-a011f5e841f2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:15.985812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:15.989167 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:15.989895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:15.990142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:15.990422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:15.991150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:15.992760 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.012305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1212/2405] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:16.012305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.024115 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28a97f-ad9b-419b-94df-ad445b01427f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.029400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.030292 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.035614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.035614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.035614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.035614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.051074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1213/2406] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:16.055335 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.074569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1194/2407] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:15 2022] POST /identity/v3/users => generated 309 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:16.074680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b339-aff8-45a4-b7fb-fae884df3b57 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.075897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.083086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.084227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.084668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.085116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] PATH_INFO: `/v3/projects/0eb0f03ff91b4071a92d9c8c1e459f40/users/3156bf18539b492daa8ab3ebd9da9f23/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.098495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27266c57-6e80-4dc1-a76a-01713a7f358c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.109478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.110753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb0f03ff91b4071a92d9c8c1e459f40, user_id=3156bf18539b492daa8ab3ebd9da9f23, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.111399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.111988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.112373 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.114134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.114557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46f59-0a04-4ed3-a9c8-254130f3d4c4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.118392 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.140186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1195/2408] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:16.143691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.155591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a505c-2306-45e6-a45d-11ca7e859e7e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.162541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.163934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.164632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.165033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] PATH_INFO: `/v3/projects/3b693253d87b491b86824bec82a263d2/users/db69d9e4b6f540a188de99a7ab27d4c4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.185842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b693253d87b491b86824bec82a263d2, user_id=db69d9e4b6f540a188de99a7ab27d4c4, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.193215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.197550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1214/2409] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:16 2022] PUT /identity/v3/projects/0eb0f03ff91b4071a92d9c8c1e459f40/users/3156bf18539b492daa8ab3ebd9da9f23/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:16.208929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.209311 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.209529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.260603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] MFA Rules not processed for user `3156bf18539b492daa8ab3ebd9da9f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:16.262345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:16.288141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:16.299624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1196/2410] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:16 2022] PUT /identity/v3/projects/3b693253d87b491b86824bec82a263d2/users/db69d9e4b6f540a188de99a7ab27d4c4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:16.301578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.339393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:16.379927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccbe2a-a9ad-4a92-b911-1f983c9bda3b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.383285 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1215/2411] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:16.389110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.389862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.390187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.394146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.394146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.394146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.396469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.398042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:16.406173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1197/2412] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:16.410281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.423725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba5e9a-7747-44cd-a8e1-a626e53c87c0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.430069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.431329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.431462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.431901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] PATH_INFO: `/v3/projects/3b693253d87b491b86824bec82a263d2/users/db69d9e4b6f540a188de99a7ab27d4c4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.435726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b693253d87b491b86824bec82a263d2, user_id=db69d9e4b6f540a188de99a7ab27d4c4, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.439079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d36a46-fb7d-40ed-a3ad-d0c577e7d55b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.453750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1198/2413] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:16 2022] PUT /identity/v3/projects/3b693253d87b491b86824bec82a263d2/users/db69d9e4b6f540a188de99a7ab27d4c4/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:16.464151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.464475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.464862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.499764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7988db1-4a86-4f91-b679-8cd0b71d859e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.500043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] MFA Rules not processed for user `db69d9e4b6f540a188de99a7ab27d4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:16.501200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:16.570327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.571050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.571259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.571580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.599212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1199/2414] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:16.610516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.616105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:16.643371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47344122-bf12-4a35-95fa-dfe8654b2686 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.653100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.654767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.658579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.659188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.659472 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.659747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.667748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1216/2415] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:16.677182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.677805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd836914-6e03-49e5-9aa2-e612fdf83156 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:16.689654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1200/2416] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:16.864884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:16.920689 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:16.969491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd455856-1644-420d-9964-999fd3776b25 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:16.975814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:16.976898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:16.977262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:16.977626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] PATH_INFO: `/v3/users/a256c5b8d1464727ab4e31539c999ec0` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:16.979391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a256c5b8d1464727ab4e31539c999ec0)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:16.981762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:17.152970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a256c5b8d1464727ab4e31539c999ec0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:17.153693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a256c5b8d1464727ab4e31539c999ec0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:17.154340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a256c5b8d1464727ab4e31539c999ec0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:17.172015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a256c5b8d1464727ab4e31539c999ec0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:17.180038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a256c5b8d1464727ab4e31539c999ec0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:17.247920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1217/2417] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:16 2022] DELETE /identity/v3/users/a256c5b8d1464727ab4e31539c999ec0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:17.252907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:17.253860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:17.254200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:17.281287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:17.282970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:17.385860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1201/2418] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:17.396922 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:17.399383 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:17.486422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499dea5-3873-422d-a28d-0e5c8677c40d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:17.490955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:17.491741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:17.492075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:17.492408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:17.505995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:17.510809 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:17.529897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1218/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:18.020962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:18.022574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:18.036964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c3d1f-a238-49be-8525-d8d4e461ea35 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:18.053138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:18.054433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:18.054721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:18.055017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] PATH_INFO: `/v3/projects/b0d06240acd94a6ebf3d9b4b1048a6b8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:18.057081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d06240acd94a6ebf3d9b4b1048a6b8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:18.059123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:18.080146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:18.162927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d06240acd94a6ebf3d9b4b1048a6b8', 'request_id': 'req-d5246c08-7952-4604-b486-1fd1a5351d69'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.259700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d06240acd94a6ebf3d9b4b1048a6b8', 'request_id': 'req-d5246c08-7952-4604-b486-1fd1a5351d69'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.267467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d06240acd94a6ebf3d9b4b1048a6b8', 'request_id': 'req-d5246c08-7952-4604-b486-1fd1a5351d69'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.282097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] The token cache is being invalidate because project b0d06240acd94a6ebf3d9b4b1048a6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:18.282339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.282955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.288281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1202/2420] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:18 2022] DELETE /identity/v3/projects/b0d06240acd94a6ebf3d9b4b1048a6b8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:18.289947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:18.300344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10af84-2719-4be0-af95-d8ef0de9a497 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:18.359596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:18.372288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:18.373457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:18.373916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:18.374371 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] PATH_INFO: `/v3/users/db69d9e4b6f540a188de99a7ab27d4c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:18.380843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=db69d9e4b6f540a188de99a7ab27d4c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:18.385344 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:18.591168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db69d9e4b6f540a188de99a7ab27d4c4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.591830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db69d9e4b6f540a188de99a7ab27d4c4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.592503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db69d9e4b6f540a188de99a7ab27d4c4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.607798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db69d9e4b6f540a188de99a7ab27d4c4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.616124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db69d9e4b6f540a188de99a7ab27d4c4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:18.689036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1219/2421] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:18 2022] DELETE /identity/v3/users/db69d9e4b6f540a188de99a7ab27d4c4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:18.988346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:18.992585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:19.106410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5246c08-7952-4604-b486-1fd1a5351d69 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:19.187952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:19.188695 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.189219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.189591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.194271 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:19.197477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:19.200069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:19.313647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149cb9-7af9-40b9-bc8a-41f5ba058bcc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:19.319267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:19.320170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.320498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.320773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] PATH_INFO: `/v3/projects/3b693253d87b491b86824bec82a263d2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.322039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b693253d87b491b86824bec82a263d2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:19.324517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:19.342443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:19.344157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe60154-d21e-42d2-8bed-7bcf5c6ab81e service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:19.355603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1203/2422] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:19.447064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b693253d87b491b86824bec82a263d2', 'request_id': 'req-4c055391-374b-4dc7-b9d1-43dddb437b92'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:19.527592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b693253d87b491b86824bec82a263d2', 'request_id': 'req-4c055391-374b-4dc7-b9d1-43dddb437b92'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:19.537022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b693253d87b491b86824bec82a263d2', 'request_id': 'req-4c055391-374b-4dc7-b9d1-43dddb437b92'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:19.558165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:19.568165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] The token cache is being invalidate because project 3b693253d87b491b86824bec82a263d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:19.568549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:19.569417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c055391-374b-4dc7-b9d1-43dddb437b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:19.574575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1220/2423] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:19 2022] DELETE /identity/v3/projects/3b693253d87b491b86824bec82a263d2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:19.576586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.576955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.577310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.600872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:19.602498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:19.618070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:19.704287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1204/2424] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:19.711861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.712596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.712882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.741308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:19.742347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:19.753286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1221/2425] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:19.758529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:19.769068 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf50f3-391d-435c-b8e1-0ef3ab386965 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:19.780238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:19.780928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.781208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.781480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.782247 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:19.782553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:19.784192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:19.802351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1205/2426] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:29:19.803053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:19.820171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742acee4-4a47-4c1a-aa6a-a9b9f148c361 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:19.825961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:19.826695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.827073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.827314 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.828094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:19.830043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:19.900899 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1222/2427] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:19 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:19.904944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:19.918814 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0078b-37c2-4263-b410-bc28ae56fea6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:19.990346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:19.991217 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:19.991589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:19.991943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:19.992701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:19.994719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:20.096590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1206/2428] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:19 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:20.100647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:20.114229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f19a2-0f3d-462b-b0fe-f105ae57500c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:20.128476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:20.130060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:20.130623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:20.131591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:20.132929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:20.135084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:20.154124 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1223/2429] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:20.157857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:20.170817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc70db-250c-4c67-b4bd-5000f27e06eb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:20.178463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:20.179624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:20.180191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:20.180701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:20.181802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:20.184628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425e54f-d722-4d82-bf34-b16c067779cd admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:20.201491 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1207/2430] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:20.205839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:20.223521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1de2b-cc5d-4ed8-aa07-a0698c0d5f6a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:20.235254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:20.236324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:20.236672 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:20.237017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] PATH_INFO: `/v3/projects/3ab00b40a4bd471da8021231def9495b/users/ce6f276463fa43d09dd8793b90bd6359/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:20.255471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab00b40a4bd471da8021231def9495b, user_id=ce6f276463fa43d09dd8793b90bd6359, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:20.258966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:20.321254 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1224/2431] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:20 2022] PUT /identity/v3/projects/3ab00b40a4bd471da8021231def9495b/users/ce6f276463fa43d09dd8793b90bd6359/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:20.323687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93bf67-0791-43ee-b493-7e2150f7aef0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:20.324138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93bf67-0791-43ee-b493-7e2150f7aef0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:20.324629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93bf67-0791-43ee-b493-7e2150f7aef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:20.351039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1c93bf67-0791-43ee-b493-7e2150f7aef0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:20.353400 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1c93bf67-0791-43ee-b493-7e2150f7aef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:20.427576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1208/2432] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:20.437986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:20.439860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:20.502918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635a8a8-1ec1-43ae-b19a-6f8b0c79e813 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:20.506762 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:20.507439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:20.507722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:20.508000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:20.518224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:20.519757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:20.535637 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1225/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:22.421471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:22.424750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:22.424750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:22.454928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] MFA Rules not processed for user `ce6f276463fa43d09dd8793b90bd6359`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:22.456434 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:22.463627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:22.562903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1209/2434] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:22.564185 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:22.578274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf5075-7a2d-41df-a517-fefcbd0bd34d service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:22.587430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:22.587430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:22.587430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:22.587430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:22.587805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:22.588863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:22.711664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1226/2435] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:22 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:22.716229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:22.739740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780555f-23c6-4674-8296-6f7995bd240a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:22.840899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:22.841738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:22.847305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:22.847305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:22.847305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:22.847305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:22.938664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1210/2436] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:22 2022] POST /identity/v3/users => generated 309 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:22.939708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:22.961337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e353e6-d8a2-49ab-a3a9-6a9d2e843a86 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:22.970384 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:22.971078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:22.971392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:22.971658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:22.972390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:22.974173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:22.994777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1227/2437] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:22.996154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:23.014904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcd00-2bb8-4f4c-ba6c-6856e5c5c7b7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:23.020645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:23.021881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:23.022427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:23.022973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] PATH_INFO: `/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9/users/5e4e3e268fb941ea8611103c50c1bc31/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:23.042857 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7966a69ec9aa42a996e7f335fa4c6bc9, user_id=5e4e3e268fb941ea8611103c50c1bc31, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:23.048526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:23.149145 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1211/2438] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:22 2022] PUT /identity/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9/users/5e4e3e268fb941ea8611103c50c1bc31/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:23.153071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:23.163497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc50ca-6e2a-4760-a5f2-ae7ceaeee72a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:23.229765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:23.239436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:23.240515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:23.241130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:23.241572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:23.242492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:23.244390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:23.269743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1228/2439] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:23.269958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:23.279758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ffd0d-3c1e-4e1e-9a8c-5ca1d285883a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:23.285928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:23.287169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:23.287453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:23.287735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] PATH_INFO: `/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9/users/5e4e3e268fb941ea8611103c50c1bc31/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:23.289822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7966a69ec9aa42a996e7f335fa4c6bc9, user_id=5e4e3e268fb941ea8611103c50c1bc31, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:23.292598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:23.317516 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1212/2440] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:23 2022] PUT /identity/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9/users/5e4e3e268fb941ea8611103c50c1bc31/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:24.598158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:24.606349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:24.718116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:24.719982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:24.737430 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920f01b-e6d4-42be-b832-53fa5ec1f145 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:24.804683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170ef27-0a8e-4cf5-85e0-7c4877fc6c6a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:24.809378 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:24.810289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:24.810579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:24.810855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] PATH_INFO: `/v3/users/954933c1cb0249c2a86b309b6f8a334c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:24.812112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=954933c1cb0249c2a86b309b6f8a334c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:24.814057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:24.817005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:24.820774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:24.821427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:24.821712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:24.821983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:24.824074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:24.835347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:24.836941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58a811-873c-4088-ad02-6abdfcfb1c72 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:24.850583 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1229/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:25.041370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954933c1cb0249c2a86b309b6f8a334c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.042195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954933c1cb0249c2a86b309b6f8a334c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.042881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954933c1cb0249c2a86b309b6f8a334c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.056536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954933c1cb0249c2a86b309b6f8a334c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.063904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954933c1cb0249c2a86b309b6f8a334c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.132920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1213/2442] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:24 2022] DELETE /identity/v3/users/954933c1cb0249c2a86b309b6f8a334c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:25.239289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af9ee-c8ae-445b-b9e1-344c826d665f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.239595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af9ee-c8ae-445b-b9e1-344c826d665f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.239879 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af9ee-c8ae-445b-b9e1-344c826d665f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.272874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c44af9ee-c8ae-445b-b9e1-344c826d665f None None] MFA Rules not processed for user `5e4e3e268fb941ea8611103c50c1bc31`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:25.274168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c44af9ee-c8ae-445b-b9e1-344c826d665f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:25.359791 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1230/2443] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:25 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:25.444776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:25.446588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:25.564395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e764ea-e88c-431e-ab9e-127ced163d6a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:25.569873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:25.570807 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.571123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.571420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] PATH_INFO: `/v3/projects/cb1d9be9203346cd8cf364305116abf1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.573017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1d9be9203346cd8cf364305116abf1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:25.575248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:25.698650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1d9be9203346cd8cf364305116abf1', 'request_id': 'req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.773125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1d9be9203346cd8cf364305116abf1', 'request_id': 'req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.787028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1d9be9203346cd8cf364305116abf1', 'request_id': 'req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.796903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] The token cache is being invalidate because project cb1d9be9203346cd8cf364305116abf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:25.796903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.797909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b0d13fa3-9cb0-4550-9597-c1c60fe4d4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:25.800262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1214/2444] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:25 2022] DELETE /identity/v3/projects/cb1d9be9203346cd8cf364305116abf1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:25.805036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.805519 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.805769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.829547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:25.829547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:25.835133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:25.895997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1231/2445] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:25.905070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.905669 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.906200 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.926978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:25.930402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:25.936751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1215/2446] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:25.941743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:25.956719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb135ad4-47e6-4c82-b33e-ec92bfea79de None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:25.958442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:25.959225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.959611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.959982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.960831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:25.961272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:25.963190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:25.979054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1232/2447] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:29:25.980227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:25.984911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:25.991114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b781b-4647-49dd-abd4-801a8a2af020 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:25.995904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:25.996565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:25.996848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:25.997143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:25.997812 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:25.999757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.131775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1216/2448] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:25 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:26.136460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.147127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd5c0-a84d-412a-af70-b738abe0f5f0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.200863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.203346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.223585 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.224578 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.227436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.227912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.228810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.230983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.299924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776929-144b-4d09-a03a-b67c1e234952 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.309739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.309739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.309739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.309739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] PATH_INFO: `/v3/users/3156bf18539b492daa8ab3ebd9da9f23` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.310973 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3156bf18539b492daa8ab3ebd9da9f23)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.313069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.314522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1233/2449] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:26 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:26.315558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.326878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85989fc-19a1-4045-8e1c-ce587e957103 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.331719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.332429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.338286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.338286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.338286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.338286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.346432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1234/2450] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:26.350752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.360116 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b15a-5ea9-4772-a67f-10a1dbfff823 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.365556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.366225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.366508 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.366781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.367485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.369048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.380251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1235/2451] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:26.384392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.393656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699558-c280-4267-b7ba-bcbc2df1c5ba admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.399931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.400811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.401093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.401368 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] PATH_INFO: `/v3/projects/30b1720707154211b22767e531bd55b1/users/de2a92702fa34d0bbf180362c39051de/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.419359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b1720707154211b22767e531bd55b1, user_id=de2a92702fa34d0bbf180362c39051de, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.422122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b15e6-9bdc-43e5-85df-758bb5895870 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.484674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3156bf18539b492daa8ab3ebd9da9f23'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:26.485294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3156bf18539b492daa8ab3ebd9da9f23'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:26.486320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3156bf18539b492daa8ab3ebd9da9f23'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:26.494949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1236/2452] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:26 2022] PUT /identity/v3/projects/30b1720707154211b22767e531bd55b1/users/de2a92702fa34d0bbf180362c39051de/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:26.504988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.505268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.505546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.508126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3156bf18539b492daa8ab3ebd9da9f23'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:26.515806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a3c7e86-2874-4b5c-948d-c52fd6a2c685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3156bf18539b492daa8ab3ebd9da9f23'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:26.535850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] MFA Rules not processed for user `de2a92702fa34d0bbf180362c39051de`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:26.537133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.566731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:26.584978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1217/2453] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:26 2022] DELETE /identity/v3/users/3156bf18539b492daa8ab3ebd9da9f23 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:26.589242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.589939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.590341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.611483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:26.612711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.613648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1237/2454] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:26.614658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.616070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.685160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1218/2455] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:26.693195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af440a2b-16f5-47d5-b55e-580d5ae7f89d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.697848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.699517 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.699650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.700260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.700547 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.700931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.701745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.705929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.759125 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325f15e-3b57-4f7a-8942-1ccf503a9ff2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.762639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.763288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.763570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.763846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.773210 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.774674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.787424 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1219/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:26.791361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.792755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:26.805944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85d769-6697-418d-837e-04772e6b0aa7 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.811190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.812151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.812512 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.812847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] PATH_INFO: `/v3/users/ce6f276463fa43d09dd8793b90bd6359` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.814212 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6f276463fa43d09dd8793b90bd6359)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.816319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:26.819912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1238/2457] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:26 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:26.821328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:26.886645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28b18-32a0-4648-8d41-8f64f12ae9bd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:26.891821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:26.892498 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:26.892783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:26.893058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:26.893743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:26.896237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.008299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1239/2458] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:26 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:27.012470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.024057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5703e-d766-48ec-9605-c371e284aaa5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.029302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.030090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.030416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.030766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.031565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.033306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.050360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1240/2459] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:27.051799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.052351 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6f276463fa43d09dd8793b90bd6359'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.053237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6f276463fa43d09dd8793b90bd6359'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.053993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6f276463fa43d09dd8793b90bd6359'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.063354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d437719-283a-46a5-b6ce-7e67f87dfc74 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.070041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.070498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6f276463fa43d09dd8793b90bd6359'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.071184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.071566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.072241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] PATH_INFO: `/v3/projects/42ea89f42e3a482091573a57cef0fb7b/users/54af2f83c17f4df6b8335c34ef3fd6fa/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.082313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6f276463fa43d09dd8793b90bd6359'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.094608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ea89f42e3a482091573a57cef0fb7b, user_id=54af2f83c17f4df6b8335c34ef3fd6fa, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.097341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.146733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1220/2460] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:26 2022] DELETE /identity/v3/users/ce6f276463fa43d09dd8793b90bd6359 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:27.164546 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1241/2461] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:27 2022] PUT /identity/v3/projects/42ea89f42e3a482091573a57cef0fb7b/users/54af2f83c17f4df6b8335c34ef3fd6fa/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:27.168237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.169694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.225092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.227406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.247075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06305-46e4-4be7-9622-22fce40b08b5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.247414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:27.309395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.310114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.310403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.310688 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.311427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.313050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.322428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:27.332945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1221/2462] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:27.332945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.336843 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03daee-b61a-462e-b888-33b608073e72 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.343200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.343408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f095-98f1-41a7-b98c-3b8713f688d0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.344960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.344960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.345897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] PATH_INFO: `/v3/projects/0eb0f03ff91b4071a92d9c8c1e459f40` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.347957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb0f03ff91b4071a92d9c8c1e459f40)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.349987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.353742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.353742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.353742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.353742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] PATH_INFO: `/v3/projects/42ea89f42e3a482091573a57cef0fb7b/users/54af2f83c17f4df6b8335c34ef3fd6fa/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.355634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ea89f42e3a482091573a57cef0fb7b, user_id=54af2f83c17f4df6b8335c34ef3fd6fa, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.358323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6bfc6-a4c5-4677-b0ee-4efed023572f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.371152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1222/2463] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:27 2022] PUT /identity/v3/projects/42ea89f42e3a482091573a57cef0fb7b/users/54af2f83c17f4df6b8335c34ef3fd6fa/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:27.381946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.382272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.384254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.413184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] MFA Rules not processed for user `54af2f83c17f4df6b8335c34ef3fd6fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:27.414430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.477995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb0f03ff91b4071a92d9c8c1e459f40', 'request_id': 'req-86424c78-48b9-42d6-8cf2-2700501e1b9d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.490304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1223/2464] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:27.490304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.495342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.551306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb0f03ff91b4071a92d9c8c1e459f40', 'request_id': 'req-86424c78-48b9-42d6-8cf2-2700501e1b9d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.558163 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb0f03ff91b4071a92d9c8c1e459f40', 'request_id': 'req-86424c78-48b9-42d6-8cf2-2700501e1b9d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.569747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d928b-503b-4095-8890-e8deef0ae90f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.570844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] The token cache is being invalidate because project 0eb0f03ff91b4071a92d9c8c1e459f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:27.571137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.571833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.573807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1242/2465] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:27 2022] DELETE /identity/v3/projects/0eb0f03ff91b4071a92d9c8c1e459f40 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:27.574647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.578459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.638260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.639149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.639435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.639738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] PATH_INFO: `/v3/projects/3ab00b40a4bd471da8021231def9495b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.639926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86424c78-48b9-42d6-8cf2-2700501e1b9d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.641113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab00b40a4bd471da8021231def9495b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.643155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.643929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.644678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.645029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.645608 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.647496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.711459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.712895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfebd1-4ec9-4e59-9fcc-1f7131514bf5 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.722643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1243/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:27.723939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fcae-f5a4-4982-b8e4-586c37326068 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.724227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fcae-f5a4-4982-b8e4-586c37326068 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.724500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6fcae-f5a4-4982-b8e4-586c37326068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.742798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-16f6fcae-f5a4-4982-b8e4-586c37326068 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:27.744201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-16f6fcae-f5a4-4982-b8e4-586c37326068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.745452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab00b40a4bd471da8021231def9495b', 'request_id': 'req-6666ce87-9aae-4711-87f4-ed216788ee97'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.802806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1244/2467] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:27.810463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.810754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.811497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.812701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab00b40a4bd471da8021231def9495b', 'request_id': 'req-6666ce87-9aae-4711-87f4-ed216788ee97'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.820270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab00b40a4bd471da8021231def9495b', 'request_id': 'req-6666ce87-9aae-4711-87f4-ed216788ee97'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.832485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] The token cache is being invalidate because project 3ab00b40a4bd471da8021231def9495b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:27.832778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.833443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:27.835278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1224/2468] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:27 2022] DELETE /identity/v3/projects/3ab00b40a4bd471da8021231def9495b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:27.838412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:27.838947 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.839657 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.840777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:27.849288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1245/2469] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:27.853125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:27.854305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.855317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce87-9aae-4711-87f4-ed216788ee97 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.867437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.867437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.867437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.867437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] PATH_INFO: `/v3/users/5e4e3e268fb941ea8611103c50c1bc31` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.867854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4e3e268fb941ea8611103c50c1bc31)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.867943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b25cd-30a5-443a-9146-3941b5fed3b0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.868059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.868822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.869581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.869900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.870210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.870945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:27.871385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.873086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:27.884983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1246/2470] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:29:27.889202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:27.899053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6c1c-57ea-4eb4-9fef-889700c2394f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:27.903769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:27.904412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:27.904700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:27.904979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:27.905687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:27.907675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.037435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1247/2471] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:27 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:28.038409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.040004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.066537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4e3e268fb941ea8611103c50c1bc31'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.067256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4e3e268fb941ea8611103c50c1bc31'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.067888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4e3e268fb941ea8611103c50c1bc31'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.080394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4e3e268fb941ea8611103c50c1bc31'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.087375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4e3e268fb941ea8611103c50c1bc31'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.099175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d58d-2026-4681-b911-d3e8ec9abb68 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.103050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.103700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.103985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.104265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.106093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.121970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:28.149792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1225/2472] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:27 2022] DELETE /identity/v3/users/5e4e3e268fb941ea8611103c50c1bc31 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:28.150716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.152011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.165974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.173503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.177056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1248/2473] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:28.234389 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46497c9-4aa3-4074-a30e-a8dcfaed186c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.239146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.239867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.240162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.240444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.241140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.243104 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.361711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1226/2474] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:28 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:28.365776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.391310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.393045 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.405921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f99b8-5b38-4b7c-a2be-76e8cd485035 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.417909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.417909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.417909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.417909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] PATH_INFO: `/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.418741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7966a69ec9aa42a996e7f335fa4c6bc9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.418741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.436524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124eadb-bc62-44f9-9170-4d4ae128701a service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.441100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.445358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.445358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.445358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.445358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.445358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.454863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1249/2475] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:28.459235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.468881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2a42-a465-4f31-b14f-f3fe77e2b9d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.474182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.474827 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.475114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.475392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.476104 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.477684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.487521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1250/2476] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:28.491649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.500666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e9ec5-6620-4932-9e5b-51bf64fdd8f5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.505264 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.506130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.506417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.506697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] PATH_INFO: `/v3/projects/add3d6328c754244a17f23d21ff30fac/users/e6dc0d72c1fb4c2fb4f0b704ffafadb1/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.521286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add3d6328c754244a17f23d21ff30fac, user_id=e6dc0d72c1fb4c2fb4f0b704ffafadb1, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.530724 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc4fe3-dfeb-402b-86ca-d7a3ccf43c95 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.642885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1251/2477] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:28 2022] PUT /identity/v3/projects/add3d6328c754244a17f23d21ff30fac/users/e6dc0d72c1fb4c2fb4f0b704ffafadb1/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:28.648597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.648893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.649172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.654780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7966a69ec9aa42a996e7f335fa4c6bc9', 'request_id': 'req-6f26bb65-91da-476a-b47c-5a28822edf8b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.667577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:28.668786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.728602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1252/2478] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:28.738811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7966a69ec9aa42a996e7f335fa4c6bc9', 'request_id': 'req-6f26bb65-91da-476a-b47c-5a28822edf8b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.739691 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.741062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.745261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7966a69ec9aa42a996e7f335fa4c6bc9', 'request_id': 'req-6f26bb65-91da-476a-b47c-5a28822edf8b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.757454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] The token cache is being invalidate because project 7966a69ec9aa42a996e7f335fa4c6bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:28.757768 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.758698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f26bb65-91da-476a-b47c-5a28822edf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:28.760388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1227/2479] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:28 2022] DELETE /identity/v3/projects/7966a69ec9aa42a996e7f335fa4c6bc9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:28.765040 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68e81-ba52-4467-a7a6-ebf82e46a7c0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.765329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68e81-ba52-4467-a7a6-ebf82e46a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.765657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68e81-ba52-4467-a7a6-ebf82e46a7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.774293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:28.785233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6fd68e81-ba52-4467-a7a6-ebf82e46a7c0 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:28.786458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6fd68e81-ba52-4467-a7a6-ebf82e46a7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.795949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1228/2480] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:28.801567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:28.803600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.803888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.804173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.807606 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844940ca-0c05-4f14-b874-63fe8f9b46c5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.811453 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.812101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.812385 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.812660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.814477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.822302 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:28.823743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:28.827267 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.828698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.832645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1229/2481] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:28.836538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.839165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1253/2482] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:28.849299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeadab-808b-4fd9-8d95-4287ff995fe5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.850113 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.850767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.851048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.851324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.852043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:28.852903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.853925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.866057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1230/2483] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:29:28.870560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:28.880731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f264-1003-4b56-90d3-ac3c0f55f852 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:28.885852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:28.886571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:28.886857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:28.887133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:28.887885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:28.889812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:28.894903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:28.967806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:29.004618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1254/2484] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:28 2022] POST /identity/v3/projects => generated 407 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:29.008708 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:29.018956 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077763-2c90-488b-a856-0a501c7a0b56 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:29.075048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:29.075904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:29.076196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:29.076473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:29.077184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:29.079132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:29.220354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1231/2485] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:29 2022] POST /identity/v3/users => generated 304 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:29.227053 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:29.237466 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df318fec-afa8-49d9-895b-4f7a60a63129 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:29.242548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:29.243217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:29.243506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:29.243776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:29.244496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:29.246205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:29.258074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1255/2486] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:29.261835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:29.272731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb0e17-a8e2-469b-aea5-237dd690693c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:29.290360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:29.291277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:29.291628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:29.291969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:29.292798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:29.294505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180667a-4a6f-4a14-a918-e84e93b1ab58 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:29.312624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1232/2487] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:29.316961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:29.335004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85f9f0-9e27-4c66-b774-f5dac2ebf3c0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:29.339521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:29.340413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:29.340693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:29.340964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] PATH_INFO: `/v3/projects/f00718027c6b4edfa3ce405686ff3aca/users/6fd4820d40df44aaa4e5d91dcc9c2920/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:29.361736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00718027c6b4edfa3ce405686ff3aca, user_id=6fd4820d40df44aaa4e5d91dcc9c2920, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:29.365781 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.219319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1256/2488] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:29 2022] PUT /identity/v3/projects/f00718027c6b4edfa3ce405686ff3aca/users/6fd4820d40df44aaa4e5d91dcc9c2920/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:30.230613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.231055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.231172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.259420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] MFA Rules not processed for user `6fd4820d40df44aaa4e5d91dcc9c2920`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:30.260628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:30.301835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:30.328432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1233/2489] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:30.333949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.401289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65d20-26ca-4873-8336-401ce9d244b4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.406754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.409111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.409111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.409111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.409111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.411281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.482338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1257/2490] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:30 2022] POST /identity/v3/projects => generated 407 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:30.482852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.494318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d527-0c30-49d6-9433-4b1fd6b4d167 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.559588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.560310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.560612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.560918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.561594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.563591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.646528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1234/2491] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:30 2022] POST /identity/v3/users => generated 310 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:30.646722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.656627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add35a-4be4-4da4-91da-eb74e3a71594 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.661378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.662064 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.662348 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.662716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.663413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.665019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.686758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1258/2492] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:30.689656 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.700715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba031-e8f8-4c36-91bb-ba2b364f19e5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.706173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.707115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.708035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.708321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] PATH_INFO: `/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3/users/d99d08e89d184a128b131ef40d40e175/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.726180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a1bf1eb4de4ee6b12a98c94e98f0c3, user_id=d99d08e89d184a128b131ef40d40e175, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.728986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.812196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1235/2493] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:30 2022] PUT /identity/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3/users/d99d08e89d184a128b131ef40d40e175/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:30.813478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.824939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5f47-3bde-4579-aab5-4a72ab9e90bc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.887164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.887938 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.888379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.888812 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.890043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.892671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6108cce-62f0-4913-a96f-93f80f4d4afa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.904666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1259/2494] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:30.907938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:30.919130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6cad5-7a40-4a36-bb6d-ce29655d8988 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:30.923853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:30.924741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.925022 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.926087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] PATH_INFO: `/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3/users/d99d08e89d184a128b131ef40d40e175/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:30.928084 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a1bf1eb4de4ee6b12a98c94e98f0c3, user_id=d99d08e89d184a128b131ef40d40e175, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:30.930988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:30.954370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1236/2495] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:30 2022] PUT /identity/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3/users/d99d08e89d184a128b131ef40d40e175/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:30.960491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6f5-24b3-4fd3-aaf6-57646a7adbd7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:30.960783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6f5-24b3-4fd3-aaf6-57646a7adbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:30.961061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf6f5-24b3-4fd3-aaf6-57646a7adbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:31.001892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-96adf6f5-24b3-4fd3-aaf6-57646a7adbd7 None None] MFA Rules not processed for user `d99d08e89d184a128b131ef40d40e175`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:31.003458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-96adf6f5-24b3-4fd3-aaf6-57646a7adbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:31.015363 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:31.087521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1260/2496] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:31.101213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:31.102952 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:31.174440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:31.179443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb58a13-b184-4a84-a4de-09182683339a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:31.183151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:31.183804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:31.184088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:31.184365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:31.193342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:31.194792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:31.205968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1237/2497] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:31.239382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:31.239858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:31.239945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:31.261834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:31.263154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:31.345591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1261/2498] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:31.358721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:31.361199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:31.437386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f83b-cdb8-49ed-82be-d1423e659e8a service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:31.443151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:31.444391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:31.444740 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:31.445096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:31.455849 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:31.458634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91df5a-5339-47f8-af43-a85c94212514 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:31.467190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1238/2499] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:29:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:29:31.697500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:31.699233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:31.714564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a58b88-fb6d-4670-920e-d0721f4b6a72 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:31.732695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:31.732695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:31.732695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:31.732695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:31.822213 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:31.826379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:31.838326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1262/2500] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:35.171500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.172288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.172784 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.206724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] MFA Rules not processed for user `e6dc0d72c1fb4c2fb4f0b704ffafadb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:35.208219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:35.281791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1239/2501] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:35.286849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:35.288288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:35.339646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:35.356230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f156dc8-4711-4021-a6c0-b0d7f83f5bbd service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:35.360939 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:35.361601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.361932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.362197 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.363495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:35.365422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:35.471273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1263/2502] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:35 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:35.475717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:35.486942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7344949-16a1-4635-88eb-9e364a54b597 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:35.542747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:35.543420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.543710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.543985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.544681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:35.546694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:35.636528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:35.646305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1240/2503] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:35 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:35.648048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:35.658794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc61-d414-42a8-9b7c-f46e9b43777d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:35.664665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:35.665402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.665752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.666145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.666948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:35.668627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:35.683302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1264/2504] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:35.687638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:35.697767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb78b0-5fcd-4e8c-90dc-f4d710de45a1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:35.702710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:35.703753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.703911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.704189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] PATH_INFO: `/v3/projects/6155dcead5db4331929493c1ac0ecfb9/users/241d9d242d8c4c7d91b7573d494f2d71/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.720468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6155dcead5db4331929493c1ac0ecfb9, user_id=241d9d242d8c4c7d91b7573d494f2d71, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:35.723778 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:35.846256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1241/2505] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:35 2022] PUT /identity/v3/projects/6155dcead5db4331929493c1ac0ecfb9/users/241d9d242d8c4c7d91b7573d494f2d71/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:35.851840 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:35.863066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af8ee-dcc9-47ed-a98c-9840174ce647 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:35.961405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:35.962901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:35.963349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:35.963867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:35.965143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:35.967750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:35.984485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1265/2506] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:35.989069 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:36.001099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62989c53-0833-4ffd-9e1f-9bc0b919f279 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:36.007082 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:36.008081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:36.008364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:36.008640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] PATH_INFO: `/v3/projects/6155dcead5db4331929493c1ac0ecfb9/users/241d9d242d8c4c7d91b7573d494f2d71/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:36.010963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6155dcead5db4331929493c1ac0ecfb9, user_id=241d9d242d8c4c7d91b7573d494f2d71, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:36.014221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2c7c7-cea5-426d-9d49-dff6f47bbf73 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:36.031641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1242/2507] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:35 2022] PUT /identity/v3/projects/6155dcead5db4331929493c1ac0ecfb9/users/241d9d242d8c4c7d91b7573d494f2d71/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:40.616066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:40.621846 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:40.650162 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eba432-618f-4bf9-be8a-aceb0338175a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:40.751988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:40.754184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:40.755509 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:40.756021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] PATH_INFO: `/v3/users/de2a92702fa34d0bbf180362c39051de` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:40.757751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2a92702fa34d0bbf180362c39051de)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:40.761642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:40.985096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2a92702fa34d0bbf180362c39051de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:40.985975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2a92702fa34d0bbf180362c39051de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:40.987986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2a92702fa34d0bbf180362c39051de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:41.006716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:41.010866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2a92702fa34d0bbf180362c39051de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:41.023668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2a92702fa34d0bbf180362c39051de'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:41.129721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1266/2508] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:40 2022] DELETE /identity/v3/users/de2a92702fa34d0bbf180362c39051de => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:41.129874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:41.129874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:41.129874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:41.151676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:41.152897 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:41.260937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1243/2509] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:41.271030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:41.272911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:41.358887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019981f0-83ec-470b-a45c-9e58bdc05c6f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:41.364629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:41.365510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:41.365710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:41.365989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:41.377473 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:41.381990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d087666-8977-40fd-9bd5-701d85f146af service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:41.411802 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1267/2510] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:42.049719 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:42.052574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.065179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda974-b8e7-42df-af8a-b6be1086cb60 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:42.070676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:42.071605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.071888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.072188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] PATH_INFO: `/v3/projects/30b1720707154211b22767e531bd55b1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:42.073743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b1720707154211b22767e531bd55b1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:42.075819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:42.202748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.203253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.203653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:42.225299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b1720707154211b22767e531bd55b1', 'request_id': 'req-673e95eb-da00-4e34-bedc-75189c191fce'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.250640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] MFA Rules not processed for user `241d9d242d8c4c7d91b7573d494f2d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:42.252027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.294518 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b1720707154211b22767e531bd55b1', 'request_id': 'req-673e95eb-da00-4e34-bedc-75189c191fce'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.302220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b1720707154211b22767e531bd55b1', 'request_id': 'req-673e95eb-da00-4e34-bedc-75189c191fce'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.313692 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] The token cache is being invalidate because project 30b1720707154211b22767e531bd55b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:42.314119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.314749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.316326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1244/2511] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:42 2022] DELETE /identity/v3/projects/30b1720707154211b22767e531bd55b1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:42.323992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:42.323992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.331215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1268/2512] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:42.344417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:42.345900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.384006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:42.413752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e95eb-da00-4e34-bedc-75189c191fce admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:42.413987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a8035-aa09-46aa-8824-3cd0bb8590b3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:42.418048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:42.419017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.419204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:42.419322 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.419563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:42.420643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.420769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.421097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] PATH_INFO: `/v3/users/54af2f83c17f4df6b8335c34ef3fd6fa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:42.421963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.422405 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=54af2f83c17f4df6b8335c34ef3fd6fa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:42.424342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:42.443725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:42.490171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:42.491618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b306c25-1d7a-435b-af07-274bf3c3e654 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:42.500696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1269/2513] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:29:42.668786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54af2f83c17f4df6b8335c34ef3fd6fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.669546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54af2f83c17f4df6b8335c34ef3fd6fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.670310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54af2f83c17f4df6b8335c34ef3fd6fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.683043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54af2f83c17f4df6b8335c34ef3fd6fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.690531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54af2f83c17f4df6b8335c34ef3fd6fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:42.757049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1245/2514] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:42 2022] DELETE /identity/v3/users/54af2f83c17f4df6b8335c34ef3fd6fa => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:42.777073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.777214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.777493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:42.797718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:42.798965 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.866719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1270/2515] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:42.880026 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:42.881450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:42.982557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a117c-56fc-4f77-9480-813ac8b52e8d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:42.991749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:42.992154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:42.992654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:42.993174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.005991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:43.008370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:43.017796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1246/2516] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:29:43.182622 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:43.184330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:43.282633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acd674-c93e-42d8-9f4e-3abebb0c11cb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:43.287486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:43.288483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.288867 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.289532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] PATH_INFO: `/v3/projects/42ea89f42e3a482091573a57cef0fb7b` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.295169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ea89f42e3a482091573a57cef0fb7b)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:43.297862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:43.426943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ea89f42e3a482091573a57cef0fb7b', 'request_id': 'req-d8dcf532-c9c6-4737-ba54-5d16da6c3723'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.497495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:43.499097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:43.506855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ea89f42e3a482091573a57cef0fb7b', 'request_id': 'req-d8dcf532-c9c6-4737-ba54-5d16da6c3723'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.518553 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ea89f42e3a482091573a57cef0fb7b', 'request_id': 'req-d8dcf532-c9c6-4737-ba54-5d16da6c3723'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.524469 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:43.525895 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c848df9-7ad7-4130-b77c-2d9a06640577 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:43.539277 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] The token cache is being invalidate because project 42ea89f42e3a482091573a57cef0fb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:43.539604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.540436 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d8dcf532-c9c6-4737-ba54-5d16da6c3723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.542537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1271/2517] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:43 2022] DELETE /identity/v3/projects/42ea89f42e3a482091573a57cef0fb7b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:43.551918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dabfc-6c8f-4744-8f6d-568675424732 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.552554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dabfc-6c8f-4744-8f6d-568675424732 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.552831 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dabfc-6c8f-4744-8f6d-568675424732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.581034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fe9dabfc-6c8f-4744-8f6d-568675424732 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:43.582315 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fe9dabfc-6c8f-4744-8f6d-568675424732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:43.634576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:43.645843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:43.647166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.647604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.648028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] PATH_INFO: `/v3/users/6fd4820d40df44aaa4e5d91dcc9c2920` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.650570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd4820d40df44aaa4e5d91dcc9c2920)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:43.652855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:43.669562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1272/2518] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:43.678399 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.678626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.678907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.712015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:43.713560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:43.722544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1273/2519] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:43.726911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:43.736594 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae2862-e7cd-46f5-8dbe-ff11329dcea6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:43.742756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:43.744114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.744539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.744927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.745785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:43.746869 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:43.749094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:43.765263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1274/2520] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:29:43.769393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:43.787204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02162c41-f8b4-4c64-930e-25e682f961eb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:43.794575 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:43.796539 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:43.796826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:43.797099 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:43.797835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:43.800440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:43.864841 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd4820d40df44aaa4e5d91dcc9c2920'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.866439 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd4820d40df44aaa4e5d91dcc9c2920'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.871928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd4820d40df44aaa4e5d91dcc9c2920'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.885668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd4820d40df44aaa4e5d91dcc9c2920'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.893213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bafa4322-700b-4ac8-8aff-904ccd367f86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd4820d40df44aaa4e5d91dcc9c2920'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:43.932819 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1275/2521] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:43 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:43.935522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:43.937631 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:44.032887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1247/2522] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:43 2022] DELETE /identity/v3/users/6fd4820d40df44aaa4e5d91dcc9c2920 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:44.039843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.040183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.045461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.046647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed36ef-6e1d-4fb0-9839-1e76a37eeb12 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.110906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:44.121577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:44.193707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.195847 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.196943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.197415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.199006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.201580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.212723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:44.223739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1248/2523] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:44.229343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.231386 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:44.312755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654fb77-4a87-4f83-8dce-173a31479a63 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.315870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1276/2524] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:43 2022] POST /identity/v3/users => generated 288 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:44.317674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.318892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.319135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.319671 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.319854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.336395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.337863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.349959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738f2cf-9fc2-4963-af45-bf6659a56109 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.350887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1249/2525] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:44.364288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.365208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.365584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.365958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.366845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.370340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.391596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1277/2526] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:44.395419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.419720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0605bb8-f313-43d5-ace9-563f5fd74b11 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.433402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.434709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.435219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.436430 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.437445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.439353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed5a87-b8c5-49a8-9eac-c642dcd8bea4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.458048 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1250/2527] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:44.462911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.481640 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152470a6-f2f5-4457-8e50-59aea6e669ce admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.490887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.492038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.492480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.493136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] PATH_INFO: `/v3/projects/ee08c76ccc81434bb008cec04f94c4fe/users/bfcfb95f719e46d5b44666c0e9a4349a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.515195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee08c76ccc81434bb008cec04f94c4fe, user_id=bfcfb95f719e46d5b44666c0e9a4349a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.519695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.591580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1278/2528] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:44 2022] PUT /identity/v3/projects/ee08c76ccc81434bb008cec04f94c4fe/users/bfcfb95f719e46d5b44666c0e9a4349a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:44.603328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.604403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.606911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.617542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:44.662379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] MFA Rules not processed for user `bfcfb95f719e46d5b44666c0e9a4349a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:44.663650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:44.785224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1251/2529] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:44.789510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.887126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064bca-0f24-46be-9143-9f1338bc31fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.892220 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.893090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.894060 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.894429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.895283 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.897319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.920199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:44.921630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:44.937222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c4570-f82e-4ccb-a987-ca6fc3b0031e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:44.942533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:44.943466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:44.943884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:44.944265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] PATH_INFO: `/v3/projects/f00718027c6b4edfa3ce405686ff3aca` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:44.945736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00718027c6b4edfa3ce405686ff3aca)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:44.948024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:44.999134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1279/2530] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:44 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:45.003230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.095328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00718027c6b4edfa3ce405686ff3aca', 'request_id': 'req-6a15ad9c-ea58-45fc-bf34-635bdebc877a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.115259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07446fb-229a-43c0-9d91-bb34d8addb46 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.167116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00718027c6b4edfa3ce405686ff3aca', 'request_id': 'req-6a15ad9c-ea58-45fc-bf34-635bdebc877a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.180471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00718027c6b4edfa3ce405686ff3aca', 'request_id': 'req-6a15ad9c-ea58-45fc-bf34-635bdebc877a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.214194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] The token cache is being invalidate because project f00718027c6b4edfa3ce405686ff3aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:45.214194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.217190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.223700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1252/2531] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:44 2022] DELETE /identity/v3/projects/f00718027c6b4edfa3ce405686ff3aca => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:45.225968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.231265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:45.287018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:45.311731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.313900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.315022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.316559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.335155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.338151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.378811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ad9c-ea58-45fc-bf34-635bdebc877a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.384941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.386118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.386670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.387035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] PATH_INFO: `/v3/users/d99d08e89d184a128b131ef40d40e175` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.388343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d99d08e89d184a128b131ef40d40e175)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.390712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.418460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:45.436891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1280/2532] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:45 2022] POST /identity/v3/users => generated 294 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:45.441390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.448343 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:45.464128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889bfb5-9e7c-4a7f-a961-35ede915c4af admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.470862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.471778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.472325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.477615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.477615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.478171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.503778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1281/2533] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:45.507876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.518854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73fb0b-5ed8-4bf9-a470-2d49219ad48d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.524349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.525295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.525576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.525855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] PATH_INFO: `/v3/projects/ae495e68f34743abb1b28701350e1755/users/3fc49f2d8761493191a49ba1e3824889/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.544407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae495e68f34743abb1b28701350e1755, user_id=3fc49f2d8761493191a49ba1e3824889, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.547132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.625955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1282/2534] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:45 2022] PUT /identity/v3/projects/ae495e68f34743abb1b28701350e1755/users/3fc49f2d8761493191a49ba1e3824889/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:45.628596 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.632100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd99d08e89d184a128b131ef40d40e175'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.632911 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd99d08e89d184a128b131ef40d40e175'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.633706 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd99d08e89d184a128b131ef40d40e175'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.653220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd99d08e89d184a128b131ef40d40e175'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.668003 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd99d08e89d184a128b131ef40d40e175'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:45.734822 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8c0d-bc09-44da-9f56-4db638b903fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.735346 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1253/2535] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:29:45 2022] DELETE /identity/v3/users/d99d08e89d184a128b131ef40d40e175 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:45.847942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.850075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.852964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.852964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.852964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.860440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1f4bc-d7d6-4023-aeb1-6d4051be05d3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.893616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1283/2536] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:29:45.894928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:45.896756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:45.917919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eab93-1861-4fb4-b09b-92d66cf3bff2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:45.927237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:45.931868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.931868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.931868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] PATH_INFO: `/v3/projects/ae495e68f34743abb1b28701350e1755/users/3fc49f2d8761493191a49ba1e3824889/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:45.947086 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae495e68f34743abb1b28701350e1755, user_id=3fc49f2d8761493191a49ba1e3824889, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:45.951277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:45.974134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1254/2537] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:45 2022] PUT /identity/v3/projects/ae495e68f34743abb1b28701350e1755/users/3fc49f2d8761493191a49ba1e3824889/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:45.978911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:45.979201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:45.979488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:46.029667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] MFA Rules not processed for user `3fc49f2d8761493191a49ba1e3824889`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:46.031457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:46.080237 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:46.189940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1284/2538] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:46.206809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:46.208739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:46.301817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5efc7c-b266-407e-a55f-d47cdde05e45 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:46.305912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:46.308398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:46.308398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:46.308398 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:46.309414 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:46.383332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:46.384832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e7016-35cf-487e-9433-dd0acf5f66ad service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:46.396585 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1255/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:46.453544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:46.458272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:46.583386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ec4d-2537-4d38-8a7d-debccda5ef3f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:46.589130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:46.590222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:46.590659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:46.591085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] PATH_INFO: `/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:46.592481 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a1bf1eb4de4ee6b12a98c94e98f0c3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:46.594801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:46.769567 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a1bf1eb4de4ee6b12a98c94e98f0c3', 'request_id': 'req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:46.829833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a1bf1eb4de4ee6b12a98c94e98f0c3', 'request_id': 'req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:46.839269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a1bf1eb4de4ee6b12a98c94e98f0c3', 'request_id': 'req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:46.853600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] The token cache is being invalidate because project 13a1bf1eb4de4ee6b12a98c94e98f0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:29:46.854044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:46.854824 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fbe13e1a-7524-41ac-a5d9-69eca2fa5b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:29:46.856832 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1285/2540] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:29:46 2022] DELETE /identity/v3/projects/13a1bf1eb4de4ee6b12a98c94e98f0c3 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:29:46.865009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:46.865577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:46.865861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:46.888980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:46.890521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:46.934901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:46.964749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1256/2541] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:46.972446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:46.972765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:46.973107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:46.994413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:46.995697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:47.005271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1286/2542] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:47.009711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.021015 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859fa27-e85b-48d9-bf84-2c15a0c42d85 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.025900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.026646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.026954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.027248 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.028131 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:29:47.028342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.030005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.046064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1257/2543] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:29:47.049068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.059945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26abbe-8ca6-487a-8ae7-d2bba8a65517 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.065194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.066524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.066806 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.067079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.067787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.069953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.443008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1287/2544] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:29:47 2022] POST /identity/v3/projects => generated 383 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:29:47.447005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.457649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a282082-9ff9-4dcf-b2d7-c19098d1eb07 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.517369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.518219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.518772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.519110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.519881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.522732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.614937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1258/2545] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:29:47 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:29:47.619524 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.629982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7159-a046-4c41-9906-88d57d550c75 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.635339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.639307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.639307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.639307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.639307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.639663 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.652717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1288/2546] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:47.657081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.672543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6da33-110e-4e72-be27-3d1a873b4ad1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.679473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.680583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.680941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.681288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.682078 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.683856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7f42-dbb2-403b-a442-5ee4afdeb91b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.698423 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1259/2547] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:29:47.702934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.716468 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76607c9a-9dd0-4950-9ed9-e8031299a6d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:47.722135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:47.723082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.723360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.723628 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] PATH_INFO: `/v3/projects/f098328fdd4a4f2f874e4c4b3411769a/users/f15bcc96e9954459be6ef3e65136b18e/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.741576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f098328fdd4a4f2f874e4c4b3411769a, user_id=f15bcc96e9954459be6ef3e65136b18e, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:47.744360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:47.808160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:47.826253 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1289/2548] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:29:47 2022] PUT /identity/v3/projects/f098328fdd4a4f2f874e4c4b3411769a/users/f15bcc96e9954459be6ef3e65136b18e/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:29:47.827705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:47.827999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:47.828288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:47.860316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] MFA Rules not processed for user `f15bcc96e9954459be6ef3e65136b18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:47.861757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:47.962261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1260/2549] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:29:47.974686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:47.976941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:48.045476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d621-6458-4c5c-9517-7f1b82242713 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:48.049739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:48.050681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:48.050963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:48.051245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:48.061228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:48.062855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb5456-7439-4953-863a-3147660f08d0 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:48.075997 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1290/2550] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:52.821781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:52.832338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5402c-5d9e-4629-9f77-b22056742f59 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:52.836194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:52.836878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:52.837161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:52.837435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:52.839367 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:52.875171 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:29:52.911777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:52.911777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:52.922076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1261/2551] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:54.244866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b837b-f6e3-4db3-b962-05d2859e0cf9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:54.245777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b837b-f6e3-4db3-b962-05d2859e0cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:54.246105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b837b-f6e3-4db3-b962-05d2859e0cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:54.263615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-214b837b-f6e3-4db3-b962-05d2859e0cf9 None None] MFA Rules not processed for user `e6dc0d72c1fb4c2fb4f0b704ffafadb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:54.264810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-214b837b-f6e3-4db3-b962-05d2859e0cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:54.317803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1291/2552] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:29:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:54.565816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:54.567284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:54.622083 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09375e-c4f7-476a-87e2-a0148c5246f9 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:29:54.625477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:29:54.626305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:54.626588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:54.626859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:54.629128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:54.638275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:29:54.639680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:29:54.647548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1262/2553] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:29:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:29:59.868990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:29:59.872051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:29:59.872051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:29:59.887629 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:29:59.888887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:29:59.947286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1292/2554] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:29:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:29:59.959693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:29:59.963712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:00.040670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323c144-9dec-418c-9cd5-4f73ffb01164 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:00.045632 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:00.046309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:00.046600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:00.046877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:00.054909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:00.056487 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7462aa7-ba81-4311-b624-3eecedb8173f service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:00.064579 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1263/2555] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:29:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:30:00.150148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:00.234061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:00.253269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83902c-6a17-4274-9b83-2ba58772fb2b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:00.261294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:00.262705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:00.263419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:00.263828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] PATH_INFO: `/v3/users/f15bcc96e9954459be6ef3e65136b18e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:00.267367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15bcc96e9954459be6ef3e65136b18e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:00.271551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:00.505784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15bcc96e9954459be6ef3e65136b18e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:00.506695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15bcc96e9954459be6ef3e65136b18e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:00.507810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15bcc96e9954459be6ef3e65136b18e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:00.523270 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15bcc96e9954459be6ef3e65136b18e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:00.531839 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15bcc96e9954459be6ef3e65136b18e'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:00.614766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1293/2556] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:00 2022] DELETE /identity/v3/users/f15bcc96e9954459be6ef3e65136b18e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:00.619564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:00.619878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:00.620156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:00.652447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:00.660733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:00.776073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:00.792617 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1264/2557] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:00.805378 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:00.807780 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:00.897484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac874b-b178-4954-95b6-5eed32f75ba9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:00.903803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:00.904723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:00.905190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:00.905632 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:00.920098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:00.922227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4004f-19e2-496b-9f62-084aac3638a4 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:00.934962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1294/2558] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:01.577742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:01.578961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:01.620300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d25f96-817f-47bc-92f3-50cec72d2adc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:01.639058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:01.639058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:01.639058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:01.639058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] PATH_INFO: `/v3/projects/f098328fdd4a4f2f874e4c4b3411769a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:01.639816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f098328fdd4a4f2f874e4c4b3411769a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:01.639816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:01.819010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f098328fdd4a4f2f874e4c4b3411769a', 'request_id': 'req-916705a0-e6ba-46d6-904c-d98ea763648b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:01.892250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f098328fdd4a4f2f874e4c4b3411769a', 'request_id': 'req-916705a0-e6ba-46d6-904c-d98ea763648b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:01.902724 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f098328fdd4a4f2f874e4c4b3411769a', 'request_id': 'req-916705a0-e6ba-46d6-904c-d98ea763648b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:01.918533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] The token cache is being invalidate because project f098328fdd4a4f2f874e4c4b3411769a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:01.918960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:01.919909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-916705a0-e6ba-46d6-904c-d98ea763648b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:01.922282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1265/2559] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:01 2022] DELETE /identity/v3/projects/f098328fdd4a4f2f874e4c4b3411769a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:01.926875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:01.927472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:01.927472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:01.951062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:01.953517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:02.034091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1295/2560] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:02.041835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.042262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.042691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.066758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:02.068186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:02.078497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1266/2561] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:02.083281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.098379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5583ff-4259-4c00-8bb2-16ca3878f377 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.113813 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.115397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.115775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.116340 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.117600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:02.118007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.120169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.132281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:02.140183 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1296/2562] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:02.148583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.166116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c3a99-e93f-41bc-b2cc-7dac263df9a5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.172358 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.173175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.173524 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.173925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.175251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.177652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.286670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1267/2563] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:02.287792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.298566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b533-dca1-4f82-9a7d-0fa1a4d0e36e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.357225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.357816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.358139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.358415 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.359111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.361073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.438702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1297/2564] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:02.440044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.450721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811519d4-8df9-4192-bdfe-fe151e0b2c19 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.459313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.460012 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.461401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.461401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.461401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.462927 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.475852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1268/2565] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:02.480029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.491306 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b880a-93c7-444d-a235-f054d55a7417 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.496413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.497111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.497450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.497728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.498562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.500406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4ecdd-1708-4a83-823c-66df5e28da0b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.511872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1298/2566] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:02.515994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.526379 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec89a3-a683-45ba-9158-700ed119bac3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.531005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.531893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.532178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.532454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] PATH_INFO: `/v3/projects/c32c4b7386a14a1db636307bc84627b0/users/b6652e7b20524b7584255f19bc66b62a/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.548138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32c4b7386a14a1db636307bc84627b0, user_id=b6652e7b20524b7584255f19bc66b62a, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.551859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.635787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1269/2567] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:02 2022] PUT /identity/v3/projects/c32c4b7386a14a1db636307bc84627b0/users/b6652e7b20524b7584255f19bc66b62a/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:02.647533 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.647825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.648193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.677676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] MFA Rules not processed for user `b6652e7b20524b7584255f19bc66b62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:02.679075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:02.743315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.745560 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:02.753397 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1299/2568] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:02.754472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.816260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503252-0426-49c0-8fa6-a17f37064840 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.820879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.821739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.822800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.823128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.825357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:02.844851 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aa172-a3e3-4860-b38e-183774557b06 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:02.850219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:02.850896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:02.851181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:02.851459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:02.852829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.854993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.863199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:02.869624 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:02.890831 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:02.892606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:02.914537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1270/2569] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:02.945545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1300/2570] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:02.955770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:02.975394 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9ef19-1b5e-4b5f-8e55-ca80fbcb0337 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.094316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.098900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.099455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.099898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.103002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.105240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:03.229422 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1271/2571] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:02 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:03.234864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:03.256885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940161e-3a72-430a-8372-b4ba39baecd0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.262992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.263698 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.264084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.264455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.265273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.267042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:03.284775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1301/2572] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:03.286051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:03.297721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813c186-66e0-4249-aac1-a0026cf56c8c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.303712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.304559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.305088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.305486 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.306381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.308118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c99822-42ab-4d4a-a140-b79b0bbe01f4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:03.323036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1272/2573] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:03.326814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:03.339257 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c15a8-5c36-4d33-b52a-497b16e365b9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.350009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.351245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.352106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.352464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] PATH_INFO: `/v3/projects/205daec7771b46b89c35d00bd76aa511/users/b48d94a7baed407e9f11015367aaf7e8/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.384500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=205daec7771b46b89c35d00bd76aa511, user_id=b48d94a7baed407e9f11015367aaf7e8, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.387703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:03.471427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1302/2574] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:03 2022] PUT /identity/v3/projects/205daec7771b46b89c35d00bd76aa511/users/b48d94a7baed407e9f11015367aaf7e8/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:03.481287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.481578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.481858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.509448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] MFA Rules not processed for user `b48d94a7baed407e9f11015367aaf7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:03.510792 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:03.589888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1273/2575] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:03.591106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:03.704754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98a59-4af3-4b06-a43b-850f9e0cfa1a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.715295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.715955 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.716235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.716516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.717204 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.719325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:03.814740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1303/2576] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:03 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:03.818576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:03.837951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25246b4-ae27-446f-b625-add6df9324b5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:03.891305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:03.912169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:03.913014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:03.913463 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:03.913909 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:03.914802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:03.917522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.006041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1274/2577] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:03 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:04.007304 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:04.019350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c15c1-28ee-4d52-aaf0-6e44d0d1452d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:04.024660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:04.025338 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.025619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.025895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.026662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:04.028395 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.044368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:04.058507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1304/2578] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:04.061344 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a305b9-4a40-48ac-9f5f-4b636286ab40 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:04.072638 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:04.073732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.074028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.074329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] PATH_INFO: `/v3/projects/fa63bbcbfd7640e88954e1621db19941/users/ec5b8678e787417d81d5ccb4915f7c4f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.092143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa63bbcbfd7640e88954e1621db19941, user_id=ec5b8678e787417d81d5ccb4915f7c4f, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:04.094948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.192201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1275/2579] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:04 2022] PUT /identity/v3/projects/fa63bbcbfd7640e88954e1621db19941/users/ec5b8678e787417d81d5ccb4915f7c4f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:04.196056 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:04.207341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031b94-666a-40ca-9511-8deeb8e59019 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:04.349012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:04.353556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.353844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.354233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.354998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:04.356650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475de3b2-ed81-407a-9091-b3987c61befe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.372856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1305/2580] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:04.377005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:04.390777 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f928c1-b7d3-46d1-b669-67b4075ae933 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:04.396224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:04.397123 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.397404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.397676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] PATH_INFO: `/v3/projects/fa63bbcbfd7640e88954e1621db19941/users/ec5b8678e787417d81d5ccb4915f7c4f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.399819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa63bbcbfd7640e88954e1621db19941, user_id=ec5b8678e787417d81d5ccb4915f7c4f, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:04.402533 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.413643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1276/2581] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:04 2022] PUT /identity/v3/projects/fa63bbcbfd7640e88954e1621db19941/users/ec5b8678e787417d81d5ccb4915f7c4f/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:04.426410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.426410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.426410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.459394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] MFA Rules not processed for user `ec5b8678e787417d81d5ccb4915f7c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:04.460927 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:04.509506 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:04.633255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1306/2582] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:04.646192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:04.648246 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:04.711408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f2e0-ab97-4df7-9fd8-0f0d98ce7b10 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:04.715138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:04.715817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:04.716099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:04.716370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:04.785611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:04.788306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:04.812243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1277/2583] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:07.030725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:07.032944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:07.048021 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa4faa-082f-429c-8808-f9c364c60fe5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:07.054402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:07.055627 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:07.056443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:07.056881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:07.122995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:07.124507 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:07.136126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1307/2584] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:09.313288 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:09.322631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ac1d-8402-4330-9b36-9c4ef445bcf5 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:09.326229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:09.326871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:09.327155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:09.327432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:09.337287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:09.344052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:09.348557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1278/2585] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:16.078556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:16.087962 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3c449-d879-4ccb-b4cd-93e27ae58da1 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:16.091542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:16.092610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:16.092610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:16.092815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:16.094441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:16.153411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:16.154881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3962d8-e936-490a-8622-f48134b9e62c service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:16.163675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1308/2586] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:19.344175 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:21.040725 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:21.114998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63e17-3146-4be5-bced-976500554403 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:21.120189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:21.121070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:21.121356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:21.121628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] PATH_INFO: `/v3/users/b6652e7b20524b7584255f19bc66b62a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:21.122931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6652e7b20524b7584255f19bc66b62a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:21.124933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:21.397735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6652e7b20524b7584255f19bc66b62a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:21.398540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6652e7b20524b7584255f19bc66b62a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:21.399166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6652e7b20524b7584255f19bc66b62a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:21.410994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6652e7b20524b7584255f19bc66b62a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:21.417961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6652e7b20524b7584255f19bc66b62a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:21.526504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1279/2587] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:21 2022] DELETE /identity/v3/users/b6652e7b20524b7584255f19bc66b62a => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:21.533658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:21.534178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:21.534276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:21.551778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:21.552982 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:21.600114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:21.617845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1309/2588] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:21.632382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:21.634114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:21.695161 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbdea6-9dfb-4869-b4fa-215a594adbb9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:21.699034 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:21.699665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:21.699948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:21.700215 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:21.709645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:21.711163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:21.721173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1280/2589] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:30:22.220548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:22.225100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:22.241245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f214a-2068-4b96-b177-dd648eb94304 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:22.247096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:22.248055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:22.250044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:22.250044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] PATH_INFO: `/v3/projects/c32c4b7386a14a1db636307bc84627b0` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:22.250676 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32c4b7386a14a1db636307bc84627b0)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:22.253107 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:22.426842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32c4b7386a14a1db636307bc84627b0', 'request_id': 'req-0a4305ea-65fc-4cb0-a465-0cab94e7b048'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.548859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32c4b7386a14a1db636307bc84627b0', 'request_id': 'req-0a4305ea-65fc-4cb0-a465-0cab94e7b048'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.557611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32c4b7386a14a1db636307bc84627b0', 'request_id': 'req-0a4305ea-65fc-4cb0-a465-0cab94e7b048'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.571885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] The token cache is being invalidate because project c32c4b7386a14a1db636307bc84627b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:22.572615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.573282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.575117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1310/2590] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:22 2022] DELETE /identity/v3/projects/c32c4b7386a14a1db636307bc84627b0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:22.579355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:22.589828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1ebc7-97b9-4139-ab2e-1852caec5d8b service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:22.609374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:22.623084 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:22.624537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:22.656000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:22.656000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:22.656000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:22.656000 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] PATH_INFO: `/v3/users/b48d94a7baed407e9f11015367aaf7e8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:22.656470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b48d94a7baed407e9f11015367aaf7e8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:22.657902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:22.683542 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4305ea-65fc-4cb0-a465-0cab94e7b048 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:22.687326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:22.687974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:22.688281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:22.688551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:22.690439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:22.702214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:22.703625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:22.713002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1311/2591] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:30:22.843860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b48d94a7baed407e9f11015367aaf7e8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.844586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b48d94a7baed407e9f11015367aaf7e8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.845291 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b48d94a7baed407e9f11015367aaf7e8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.858781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b48d94a7baed407e9f11015367aaf7e8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.865884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b48d94a7baed407e9f11015367aaf7e8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:22.931259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1281/2592] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:22 2022] DELETE /identity/v3/users/b48d94a7baed407e9f11015367aaf7e8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:23.345151 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:23.346751 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:23.362241 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0be7e0-2f3a-4ab1-bbf2-450face67843 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:23.425975 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:23.426882 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:23.427159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:23.427438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] PATH_INFO: `/v3/projects/205daec7771b46b89c35d00bd76aa511` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:23.428634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] RBAC: Authorizing `identity:delete_project(project_id=205daec7771b46b89c35d00bd76aa511)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:23.430686 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:23.453521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:23.593649 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205daec7771b46b89c35d00bd76aa511', 'request_id': 'req-f66dcfb3-af5d-4c13-8c54-a935b1db532b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:23.703126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205daec7771b46b89c35d00bd76aa511', 'request_id': 'req-f66dcfb3-af5d-4c13-8c54-a935b1db532b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:23.710777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205daec7771b46b89c35d00bd76aa511', 'request_id': 'req-f66dcfb3-af5d-4c13-8c54-a935b1db532b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:23.729140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] The token cache is being invalidate because project 205daec7771b46b89c35d00bd76aa511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:23.729438 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:23.730245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:23.731856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1312/2593] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:23 2022] DELETE /identity/v3/projects/205daec7771b46b89c35d00bd76aa511 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:23.735965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:23.737730 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:23.752186 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f10a-3ba8-4d19-ba33-900767296b35 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:23.818648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:23.819552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:23.819842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:23.820130 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] PATH_INFO: `/v3/users/ec5b8678e787417d81d5ccb4915f7c4f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:23.821365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5b8678e787417d81d5ccb4915f7c4f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:23.823651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:24.001178 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:24.003806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5b8678e787417d81d5ccb4915f7c4f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.004972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5b8678e787417d81d5ccb4915f7c4f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.005577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5b8678e787417d81d5ccb4915f7c4f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.017603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5b8678e787417d81d5ccb4915f7c4f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.024925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1ae1a7fe-2453-4342-b935-9695b0b2febb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5b8678e787417d81d5ccb4915f7c4f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.122474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1282/2594] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:23 2022] DELETE /identity/v3/users/ec5b8678e787417d81d5ccb4915f7c4f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:24.585611 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:24.587108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:24.599814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dcfb3-af5d-4c13-8c54-a935b1db532b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:24.669854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:24.671066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:24.671346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:24.671646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] PATH_INFO: `/v3/projects/fa63bbcbfd7640e88954e1621db19941` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:24.678558 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa63bbcbfd7640e88954e1621db19941)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:24.680707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:24.828893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa63bbcbfd7640e88954e1621db19941', 'request_id': 'req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.960940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa63bbcbfd7640e88954e1621db19941', 'request_id': 'req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.969366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa63bbcbfd7640e88954e1621db19941', 'request_id': 'req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.980804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] The token cache is being invalidate because project fa63bbcbfd7640e88954e1621db19941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:24.981085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.981759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cad48aaa-2d7a-4e86-b3d0-edcdff390fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:24.983474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1313/2595] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:24 2022] DELETE /identity/v3/projects/fa63bbcbfd7640e88954e1621db19941 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:24.989990 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:24.990498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:24.990855 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.011604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:25.013270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:25.106664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1283/2596] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:25.107561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.107907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.108190 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.135168 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:25.137744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:25.154326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.162076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1314/2597] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:25.166541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27e408-785e-4c1f-a495-3b5bf4902204 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.173772 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.174605 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.175263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.175641 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.176513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:25.176928 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.179229 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.194118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1284/2598] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:30:25.198710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.214069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be465-28de-4dda-a7ff-28dce3f34eb3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.224777 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.225609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.226231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.226613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.227480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.229666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.331252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1315/2599] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:25 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:25.335531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.351686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580f5a-01c7-493d-bc69-4a09b0ec22d7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.419761 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.428949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:25.448433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:25.497325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.498093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.498319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.498597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.499311 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.501299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.582949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70f2f7-9e39-421f-a759-2a04dd0c3734 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.593961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.595940 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.597239 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.598170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] PATH_INFO: `/v3/users/bfcfb95f719e46d5b44666c0e9a4349a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.599901 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfcfb95f719e46d5b44666c0e9a4349a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.602928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.618809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1285/2600] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:25 2022] POST /identity/v3/users => generated 287 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:25.625452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.649410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6298256-4c96-4d5b-8584-8468f09fabf2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.674583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.675298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.675586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.676418 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.677162 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.678827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.697471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1286/2601] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:25.706055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.773228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180ce51-68c5-4ef6-ab06-a57a0189a7ae admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.803907 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.804934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.807441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.807733 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.808822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.812964 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:25.837361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1287/2602] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:25.842265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:25.884769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c63135-8481-4c18-a61a-226f1c56d504 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:25.884987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfcfb95f719e46d5b44666c0e9a4349a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:25.885570 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfcfb95f719e46d5b44666c0e9a4349a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:25.886224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfcfb95f719e46d5b44666c0e9a4349a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:25.908364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:25.912335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:25.914489 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:25.914977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] PATH_INFO: `/v3/projects/a656c669853f430a9a23e5568d1adb7a/users/ae021e81bd724b8085a2e384576946dc/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:25.922155 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfcfb95f719e46d5b44666c0e9a4349a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:25.940394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-86d6e647-9235-45eb-906a-7f560108c84e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfcfb95f719e46d5b44666c0e9a4349a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:25.967258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:25.977234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a656c669853f430a9a23e5568d1adb7a, user_id=ae021e81bd724b8085a2e384576946dc, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:25.985376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675755e-9865-40cb-b999-0a08d14ac6fb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:26.029201 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1316/2603] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:25 2022] DELETE /identity/v3/users/bfcfb95f719e46d5b44666c0e9a4349a => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:26.030948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf0bc-53c9-4c8b-8a7c-e67fda6cb694 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:26.031238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf0bc-53c9-4c8b-8a7c-e67fda6cb694 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:26.031514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf0bc-53c9-4c8b-8a7c-e67fda6cb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:26.087981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8faaf0bc-53c9-4c8b-8a7c-e67fda6cb694 None None] MFA Rules not processed for user `7231b60e5f9541629fec2291c547f0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:26.087981 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8faaf0bc-53c9-4c8b-8a7c-e67fda6cb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:26.136466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1288/2604] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:25 2022] PUT /identity/v3/projects/a656c669853f430a9a23e5568d1adb7a/users/ae021e81bd724b8085a2e384576946dc/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:26.138946 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:26.140160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:26.146265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:26.189350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1317/2605] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:26.194260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:26.194740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:26.195027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:26.236279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:26.241714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:26.255919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:26.258874 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:26.387135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1289/2606] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:26.392249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1318/2607] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:26.409504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:26.409655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:26.411034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:26.411189 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:26.521678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:26.538726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3d451-cd56-4bd8-ad39-fcde6211aac7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:26.546049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13640321-4f7b-4f69-a20f-ce7d18666809 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:26.547920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:26.548659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:26.548977 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:26.549251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:26.557895 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:26.570464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:26.570464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:26.570464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:26.571553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:26.571553 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:26.577153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:26.579484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:26.599789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1319/2608] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:26.614816 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1290/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:27.964641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:27.964641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:28.011559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d0046-35a5-4f81-aeee-df77031676f1 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:28.025172 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:28.025858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:28.026336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:28.026884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] PATH_INFO: `/v3/projects/ee08c76ccc81434bb008cec04f94c4fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:28.031300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee08c76ccc81434bb008cec04f94c4fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:28.038148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:28.278610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee08c76ccc81434bb008cec04f94c4fe', 'request_id': 'req-f192444c-0896-45ff-85ce-e55ef32dab1d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.337374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee08c76ccc81434bb008cec04f94c4fe', 'request_id': 'req-f192444c-0896-45ff-85ce-e55ef32dab1d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.351401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee08c76ccc81434bb008cec04f94c4fe', 'request_id': 'req-f192444c-0896-45ff-85ce-e55ef32dab1d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.364086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] The token cache is being invalidate because project ee08c76ccc81434bb008cec04f94c4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:28.364623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.365294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f192444c-0896-45ff-85ce-e55ef32dab1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.368432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1320/2610] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:27 2022] DELETE /identity/v3/projects/ee08c76ccc81434bb008cec04f94c4fe => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:28.372862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:28.385332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67556c-57fc-4d9b-b6b5-0681c32ff6d9 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:28.466916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:28.468032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:28.468787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:28.469170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] PATH_INFO: `/v3/users/3fc49f2d8761493191a49ba1e3824889` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:28.472144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc49f2d8761493191a49ba1e3824889)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:28.474912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:28.694616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:28.705064 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc49f2d8761493191a49ba1e3824889'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.705737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc49f2d8761493191a49ba1e3824889'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.707611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc49f2d8761493191a49ba1e3824889'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.739179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc49f2d8761493191a49ba1e3824889'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.749698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc49f2d8761493191a49ba1e3824889'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:28.872967 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1291/2611] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:28 2022] DELETE /identity/v3/users/3fc49f2d8761493191a49ba1e3824889 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:29.343666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:29.344591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:29.345092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:29.379593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] MFA Rules not processed for user `ae021e81bd724b8085a2e384576946dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:29.381159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:29.474568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:29.492081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1321/2612] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:29.502839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:29.504793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:29.520242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:29.520242 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:29.580374 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01264e7-b089-4968-8a33-ce9c7f9e7c3d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:29.584847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:29.585523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:29.585810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:29.586121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:29.596627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:29.598729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6f1ca-ef7f-411e-924c-bfd62c3d38bb service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:29.609862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1292/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:29.617485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de190-76bd-4599-b7b6-88b113a4be3c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:29.623365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:29.624402 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:29.624757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:29.625101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] PATH_INFO: `/v3/projects/ae495e68f34743abb1b28701350e1755` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:29.626459 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae495e68f34743abb1b28701350e1755)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:29.628515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:29.779100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae495e68f34743abb1b28701350e1755', 'request_id': 'req-8034191f-1f03-4308-a3be-45f62bd2094a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:29.897398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae495e68f34743abb1b28701350e1755', 'request_id': 'req-8034191f-1f03-4308-a3be-45f62bd2094a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:29.904154 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae495e68f34743abb1b28701350e1755', 'request_id': 'req-8034191f-1f03-4308-a3be-45f62bd2094a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:29.915837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] The token cache is being invalidate because project ae495e68f34743abb1b28701350e1755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:29.916127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:29.916811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8034191f-1f03-4308-a3be-45f62bd2094a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:29.918417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1322/2614] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:29 2022] DELETE /identity/v3/projects/ae495e68f34743abb1b28701350e1755 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:29.923365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:29.923711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:29.924065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:29.943880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:29.945577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:30.036452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1293/2615] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:30.044386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.044682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.044967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.073143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:30.075105 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:30.101786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1323/2616] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:30.103901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.115589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701fa31-aa47-4664-b3ea-077e53e59b0b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.121117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.121810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.122128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.122450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.123359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:30.123673 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.125369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.140835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1294/2617] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:30.145433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.166674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317ee02-9da3-4b7b-ab42-0c6a7f7a7f6a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.179411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.180276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.180562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.183143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.183143 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.184654 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.296287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1324/2618] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:30 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:30.300277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.310627 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994041c8-8ba2-4ae8-ad1a-f8a8da1be8ae admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.359537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:30.394668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.395862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.396363 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.396944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.397827 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.400683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.526578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1295/2619] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:30 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:30.527653 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.538881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c6c4-af37-4b91-93d2-4b2a9cd3ae77 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.544224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.545038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.545393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.545744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.546642 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.548811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.564147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1325/2620] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:30.567006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.581752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197692f7-d17b-4c4a-a7f2-a039447c8465 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.588089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.589562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.589562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.589562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.590054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.592934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3927d16-6b36-458b-b9d3-93d626daf712 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.616214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1296/2621] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:30.627409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:30.638303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a5eea-8551-4f43-b66e-6c876042e869 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:30.643743 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:30.647551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.647551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.647551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] PATH_INFO: `/v3/projects/97b9812460c64105b2794e3dc61368cc/users/530c7f1a19eb49288e8407dc0531ccc8/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.668835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b9812460c64105b2794e3dc61368cc, user_id=530c7f1a19eb49288e8407dc0531ccc8, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:30.674142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:30.788865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1326/2622] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:30 2022] PUT /identity/v3/projects/97b9812460c64105b2794e3dc61368cc/users/530c7f1a19eb49288e8407dc0531ccc8/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:30.804118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:30.804603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:30.804886 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:30.857119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] MFA Rules not processed for user `530c7f1a19eb49288e8407dc0531ccc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:30.859213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:30.949024 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1297/2623] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:30.953625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.104173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba11a89-628c-467d-b04c-870ab232727c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.111075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.112051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.112489 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.112934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.114711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.117825 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:31.281188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1327/2624] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:30 2022] POST /identity/v3/projects => generated 381 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:31.284448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.302377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065692b-c907-4326-a1c6-58f40f6ee6f0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.391899 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.392618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.392904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.393216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.393877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.396445 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:31.505046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1298/2625] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:31 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:31.509484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.521752 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d032b-7667-4e16-96b3-467c366745bc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.529145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.529866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.530260 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.530490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.531420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.533082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c569620-96cd-4982-be52-ab381bd33df7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:31.549552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.550218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1328/2626] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:31.561604 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057da3-8461-4c57-82c4-edaefc458877 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.574728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.576614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.576614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.576614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] PATH_INFO: `/v3/projects/69b3df883f9d432d85be912e2a5eb2ab/users/894ec39a100645b89dc01711da3bedcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.579033 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.579502 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.579967 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.604356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b3df883f9d432d85be912e2a5eb2ab, user_id=894ec39a100645b89dc01711da3bedcb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.607282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:31.612599 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] MFA Rules not processed for user `ae021e81bd724b8085a2e384576946dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:31.613875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:31.625782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:31.699251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1329/2627] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:31.757868 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1299/2628] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:31 2022] PUT /identity/v3/projects/69b3df883f9d432d85be912e2a5eb2ab/users/894ec39a100645b89dc01711da3bedcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:31.764367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.789281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9af809-a38a-4edd-a6ee-d01434d1e284 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.924412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.926081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.926554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.927011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.928090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.929970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f778a-e148-40fd-a061-e0147decfeb9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:31.946823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1330/2629] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:31.951114 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:31.963856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0322e4-546d-43c2-8805-ad954a60d47b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:31.970756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:31.972070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:31.972594 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:31.973096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] PATH_INFO: `/v3/projects/69b3df883f9d432d85be912e2a5eb2ab/users/894ec39a100645b89dc01711da3bedcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:31.984060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b3df883f9d432d85be912e2a5eb2ab, user_id=894ec39a100645b89dc01711da3bedcb, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:31.984060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:32.014330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1300/2630] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:31 2022] PUT /identity/v3/projects/69b3df883f9d432d85be912e2a5eb2ab/users/894ec39a100645b89dc01711da3bedcb/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:32.020818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:32.022560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:32.025020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:32.148110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] MFA Rules not processed for user `894ec39a100645b89dc01711da3bedcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:32.149521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:32.266796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1331/2631] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:32.283315 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:32.285896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:32.290595 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:32.292903 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:32.350971 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:32.384873 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b112a5-52df-464a-b182-88a8f9f812ce None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:32.390324 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:32.392067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:32.392067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:32.392579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:32.410920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:32.413255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:32.429326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1332/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:32.480088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882421-c5ca-4be9-a2e8-edde924604e3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:32.494755 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:32.495985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:32.496280 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:32.496557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:32.503428 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:32.602242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:32.605866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:32.616837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1301/2633] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:33.300541 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:33.599132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:33.612194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f76238-2b0d-41d6-bea5-6929acf40092 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:33.619294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894ec39a100645b89dc01711da3bedcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69b3df883f9d432d85be912e2a5eb2ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:33.619565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:33.620087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:33.620548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:33.621279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:33.621758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:33.623493 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:33.638000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1333/2634] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:33.642522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:33.660427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625037bb-ff0b-4e12-a2af-33d368cbb40f service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:33.670102 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894ec39a100645b89dc01711da3bedcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69b3df883f9d432d85be912e2a5eb2ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:33.670796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:33.671077 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:33.671546 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:33.672264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:33.674222 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:33.785345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1302/2635] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:33 2022] POST /identity/v3/projects => generated 354 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:33.789587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:33.800262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8df5f-ff79-4bd9-b1d6-8099f6a248d5 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:33.890811 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894ec39a100645b89dc01711da3bedcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69b3df883f9d432d85be912e2a5eb2ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:33.891764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:33.892212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:33.892675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:33.893838 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:33.894386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:33.896907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:33.919289 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1334/2636] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:33.961652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:33.976701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce25ba3-2c0c-4f0c-bada-f3482634c27a tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:33.982734 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894ec39a100645b89dc01711da3bedcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69b3df883f9d432d85be912e2a5eb2ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:33.984089 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:33.984619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:33.985278 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] PATH_INFO: `/v3/projects/0ad176959f2c4205b0eac9397540485b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:33.987921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0ad176959f2c4205b0eac9397540485b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:33.990850 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:34.164467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad176959f2c4205b0eac9397540485b', 'request_id': 'req-4161ee47-9c4a-415b-bf48-c098f37aff55'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:34.318720 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad176959f2c4205b0eac9397540485b', 'request_id': 'req-4161ee47-9c4a-415b-bf48-c098f37aff55'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:34.338081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad176959f2c4205b0eac9397540485b', 'request_id': 'req-4161ee47-9c4a-415b-bf48-c098f37aff55'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:34.354676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] The token cache is being invalidate because project 0ad176959f2c4205b0eac9397540485b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:34.354972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:34.361652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:34.361652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1303/2637] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:33 2022] DELETE /identity/v3/projects/0ad176959f2c4205b0eac9397540485b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:34.817103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:34.817103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:34.913090 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:34.957228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a86a-6d0e-4ad8-a1de-42a1607dedee tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:35.033740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:35.034659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:35.035133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:35.035248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] PATH_INFO: `/v3/users/3be98a16abca4b5ea9e54a29f9b4ea62` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:35.036513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be98a16abca4b5ea9e54a29f9b4ea62)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:35.038571 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:35.129939 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:35.132008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:35.154815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161ee47-9c4a-415b-bf48-c098f37aff55 tempest-VolumeQuotaClassesTest-1800678805 tempest-VolumeQuotaClassesTest-1800678805-project-admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:35.171660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:35.173053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:35.173513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:35.173796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] PATH_INFO: `/v3/users/530c7f1a19eb49288e8407dc0531ccc8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:35.178144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=530c7f1a19eb49288e8407dc0531ccc8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:35.180273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:35.200893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:35.242918 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be98a16abca4b5ea9e54a29f9b4ea62'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.244016 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be98a16abca4b5ea9e54a29f9b4ea62'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.244890 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be98a16abca4b5ea9e54a29f9b4ea62'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.265885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be98a16abca4b5ea9e54a29f9b4ea62'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.278118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-95a52cbf-4eb3-4826-8221-51b9838dfef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be98a16abca4b5ea9e54a29f9b4ea62'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.360113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1335/2638] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:34 2022] DELETE /identity/v3/users/3be98a16abca4b5ea9e54a29f9b4ea62 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:35.413526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530c7f1a19eb49288e8407dc0531ccc8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.413526 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530c7f1a19eb49288e8407dc0531ccc8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.414327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530c7f1a19eb49288e8407dc0531ccc8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.443595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530c7f1a19eb49288e8407dc0531ccc8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.456124 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530c7f1a19eb49288e8407dc0531ccc8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:35.555982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1304/2639] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:35 2022] DELETE /identity/v3/users/530c7f1a19eb49288e8407dc0531ccc8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:35.560844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:35.561146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:35.561949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:35.611782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:35.613446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:35.703554 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1336/2640] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:35.716183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:35.717972 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:35.851368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddbe32-79b1-4fbe-acc9-1306b2fbfa69 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:35.870415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:35.871180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:35.871476 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:35.874588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:35.910763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:35.917324 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:35.917496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:35.921314 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:35.956335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1305/2641] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:35.970744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfba466-0923-4642-84ad-b3c33bfd4a63 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:35.980087 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:35.981486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:35.981970 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:35.982511 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] PATH_INFO: `/v3/projects/d8f120eaf89b4ef29fc92cf544ff1000` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:35.984229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f120eaf89b4ef29fc92cf544ff1000)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:35.990206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:35.993501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:36.216062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f120eaf89b4ef29fc92cf544ff1000', 'request_id': 'req-50b45ba7-4f61-4210-99f5-ae0230c51de4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.309160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f120eaf89b4ef29fc92cf544ff1000', 'request_id': 'req-50b45ba7-4f61-4210-99f5-ae0230c51de4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.326004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f120eaf89b4ef29fc92cf544ff1000', 'request_id': 'req-50b45ba7-4f61-4210-99f5-ae0230c51de4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.376907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] The token cache is being invalidate because project d8f120eaf89b4ef29fc92cf544ff1000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:36.376907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.376907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.383233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:36.390986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1337/2642] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:35 2022] DELETE /identity/v3/projects/d8f120eaf89b4ef29fc92cf544ff1000 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:36.395859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acbea-158b-4e75-ba71-c1c6c37e4c1a service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:36.500350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:36.501552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:36.501999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:36.502475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] PATH_INFO: `/v3/users/70385f22315446438a16fb69680419e9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:36.504660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=70385f22315446438a16fb69680419e9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:36.506912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:36.671892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:36.687577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70385f22315446438a16fb69680419e9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.688631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70385f22315446438a16fb69680419e9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.689741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70385f22315446438a16fb69680419e9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.706406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70385f22315446438a16fb69680419e9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.719342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70385f22315446438a16fb69680419e9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:36.785989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:36.788548 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:36.805215 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45ba7-4f61-4210-99f5-ae0230c51de4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:36.812641 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:36.813536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:36.813823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:36.814189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] PATH_INFO: `/v3/projects/97b9812460c64105b2794e3dc61368cc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:36.815419 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b9812460c64105b2794e3dc61368cc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:36.817785 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:36.834893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1306/2643] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:36 2022] DELETE /identity/v3/users/70385f22315446438a16fb69680419e9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:36.963534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b9812460c64105b2794e3dc61368cc', 'request_id': 'req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.024953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b9812460c64105b2794e3dc61368cc', 'request_id': 'req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.036446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b9812460c64105b2794e3dc61368cc', 'request_id': 'req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.057947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] The token cache is being invalidate because project 97b9812460c64105b2794e3dc61368cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:37.064679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.064679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.071626 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1338/2644] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:36 2022] DELETE /identity/v3/projects/97b9812460c64105b2794e3dc61368cc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:37.075741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:37.077329 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:37.191479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266df230-77d1-48a4-8fed-950b3121d5fc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:37.288599 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:37.290839 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:37.291478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:37.291942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] PATH_INFO: `/v3/users/894ec39a100645b89dc01711da3bedcb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:37.293347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894ec39a100645b89dc01711da3bedcb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:37.295411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:37.461535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:37.463483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:37.481816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45a775-ff5f-4104-ab12-ba0bc2f6c293 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:37.492337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:37.493406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:37.493763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:37.494195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] PATH_INFO: `/v3/projects/ef79770666e040ed97d751efd4af1a90` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:37.495514 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef79770666e040ed97d751efd4af1a90)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:37.498682 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:37.556182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894ec39a100645b89dc01711da3bedcb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.561711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894ec39a100645b89dc01711da3bedcb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.564225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894ec39a100645b89dc01711da3bedcb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.585922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:37.590722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894ec39a100645b89dc01711da3bedcb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.609051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894ec39a100645b89dc01711da3bedcb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.680337 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef79770666e040ed97d751efd4af1a90', 'request_id': 'req-3095d51a-bd71-44b5-8259-0a85795a3b48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.707466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1307/2645] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:37 2022] DELETE /identity/v3/users/894ec39a100645b89dc01711da3bedcb => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:37.772774 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef79770666e040ed97d751efd4af1a90', 'request_id': 'req-3095d51a-bd71-44b5-8259-0a85795a3b48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.782403 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef79770666e040ed97d751efd4af1a90', 'request_id': 'req-3095d51a-bd71-44b5-8259-0a85795a3b48'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.793160 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:37.808552 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] The token cache is being invalidate because project ef79770666e040ed97d751efd4af1a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:37.808941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.809896 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:37.813359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1339/2646] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:37 2022] DELETE /identity/v3/projects/ef79770666e040ed97d751efd4af1a90 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:38.689698 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:38.692035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:38.835612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716dc3b-edb7-4650-9218-94cb0676c391 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:38.849913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:38.849913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:38.849913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:38.849913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] PATH_INFO: `/v3/projects/69b3df883f9d432d85be912e2a5eb2ab` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:38.850525 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b3df883f9d432d85be912e2a5eb2ab)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:38.853163 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.019570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b3df883f9d432d85be912e2a5eb2ab', 'request_id': 'req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.100002 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.101708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:39.125110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b3df883f9d432d85be912e2a5eb2ab', 'request_id': 'req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.136912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b3df883f9d432d85be912e2a5eb2ab', 'request_id': 'req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.168008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] The token cache is being invalidate because project 69b3df883f9d432d85be912e2a5eb2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:39.168766 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.169595 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-525fb7f0-67c7-4ff0-bcab-0977d75b9b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.190793 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1308/2647] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:38 2022] DELETE /identity/v3/projects/69b3df883f9d432d85be912e2a5eb2ab => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:39.196495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362bc98-22c7-4549-ad35-376fae2e32bc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.197331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362bc98-22c7-4549-ad35-376fae2e32bc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.197554 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362bc98-22c7-4549-ad35-376fae2e32bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.232030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7362bc98-22c7-4549-ad35-376fae2e32bc None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:39.233531 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7362bc98-22c7-4549-ad35-376fae2e32bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:39.249269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095d51a-bd71-44b5-8259-0a85795a3b48 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.326139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1309/2648] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:39.334975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.335336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.335996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.358785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:39.364443 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.365716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.366275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.366784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] PATH_INFO: `/v3/users/c4586fdc3e8b4ee0ab58d8c236637a4f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.368387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4586fdc3e8b4ee0ab58d8c236637a4f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.369391 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:39.370673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.371415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:39.400536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1310/2649] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:39.402090 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.413818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204075-f2e7-42d5-82d7-c65126d706c2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.419207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.419887 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.420174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.420455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.421195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:39.421503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.423354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.437041 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1311/2650] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:39.441722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.455786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5907510-f24f-48d9-89f4-a62d1554bf7d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.462050 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.462243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.463753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.463753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.463753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.466225 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.603448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1312/2651] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:39 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:39.607619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.677916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb38d8-665d-41db-80f1-4eb7ab9d12de admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.682898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.683578 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.683859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.684164 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.684823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.686871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.707217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4586fdc3e8b4ee0ab58d8c236637a4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.707985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4586fdc3e8b4ee0ab58d8c236637a4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.708618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4586fdc3e8b4ee0ab58d8c236637a4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.721036 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4586fdc3e8b4ee0ab58d8c236637a4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.728826 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4586fdc3e8b4ee0ab58d8c236637a4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:39.784017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1340/2652] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:39 2022] DELETE /identity/v3/users/c4586fdc3e8b4ee0ab58d8c236637a4f => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:39.784745 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1313/2653] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:39 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:39.788515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.789977 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:39.817048 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:39.876063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dcbab-16c7-43a2-9d25-3b65937d7c15 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.881941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.884444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.884444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.884444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.885222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.886979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.902748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1341/2654] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:39.904014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.916866 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9faed6-5e84-4584-9b9c-a3e96565120d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.923705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.924616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.924898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.925226 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.925965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.927602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d7ac-be9d-4237-95c5-68865cc67b90 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:39.944828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1314/2655] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:39.946437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:39.959850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1904909-ab75-4836-8819-16332173e6a6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:39.967671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:39.969725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:39.970184 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:39.970458 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] PATH_INFO: `/v3/projects/a52c6692a9924a71830ccaeda5019148/users/eaa374a7f70342e592bcfb57a4ef02bb/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:39.989555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52c6692a9924a71830ccaeda5019148, user_id=eaa374a7f70342e592bcfb57a4ef02bb, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:39.993695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.070450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1342/2656] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:39 2022] PUT /identity/v3/projects/a52c6692a9924a71830ccaeda5019148/users/eaa374a7f70342e592bcfb57a4ef02bb/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:40.082881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781796a9-1911-4394-8df5-944014b29fec None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.084272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781796a9-1911-4394-8df5-944014b29fec None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.084845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781796a9-1911-4394-8df5-944014b29fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.097943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.099180 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.121769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-781796a9-1911-4394-8df5-944014b29fec None None] MFA Rules not processed for user `eaa374a7f70342e592bcfb57a4ef02bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:40.123058 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-781796a9-1911-4394-8df5-944014b29fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.197216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226aa0dd-596a-486b-8c46-48130ebae765 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.211221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.212108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.212394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.212658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] PATH_INFO: `/v3/projects/a05af8ceb6d041a6a894d5ca3ff820cc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.214885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05af8ceb6d041a6a894d5ca3ff820cc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.220858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.241787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1315/2657] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:40.242848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781796a9-1911-4394-8df5-944014b29fec None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.261277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781796a9-1911-4394-8df5-944014b29fec None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.261676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.262576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.262867 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.266538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.269920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.271737 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.422614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1316/2658] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:40.423781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.428607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05af8ceb6d041a6a894d5ca3ff820cc', 'request_id': 'req-dd00704a-600e-4caf-9627-953bf8ae4bce'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:40.494114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05af8ceb6d041a6a894d5ca3ff820cc', 'request_id': 'req-dd00704a-600e-4caf-9627-953bf8ae4bce'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:40.505778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05af8ceb6d041a6a894d5ca3ff820cc', 'request_id': 'req-dd00704a-600e-4caf-9627-953bf8ae4bce'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:40.531590 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] The token cache is being invalidate because project a05af8ceb6d041a6a894d5ca3ff820cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:40.531944 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:40.532549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-dd00704a-600e-4caf-9627-953bf8ae4bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:40.535437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1343/2659] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:40 2022] DELETE /identity/v3/projects/a05af8ceb6d041a6a894d5ca3ff820cc => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:40.540678 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e347-afd9-4f29-a7e9-3ca682ef84ac admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.541345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc5d6-5932-42dd-b650-990b8e57a9dd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.542005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc5d6-5932-42dd-b650-990b8e57a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.542921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc5d6-5932-42dd-b650-990b8e57a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.557881 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.558997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.559499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.560009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.560906 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.563453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.578878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-609cc5d6-5932-42dd-b650-990b8e57a9dd None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:40.580792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-609cc5d6-5932-42dd-b650-990b8e57a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.594101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1344/2660] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:40.600597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.601032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.601411 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.628118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:40.628118 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.636943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1345/2661] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:40.636943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.636943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.679864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1317/2662] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:40.681032 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.693788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d99b-1b0e-4e68-b51f-32116b4b91f4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.702622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.702622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.702622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.702622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.703028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:40.703735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.705335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.708989 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29510068-53a1-4d71-bd4a-3af82139183f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.719597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.720545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.720860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.721207 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.723898 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.726091 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1318/2663] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:40.727027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.729495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:40.748154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b83c3-63c1-42f5-b21c-068019ddf839 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.754929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.755621 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.755903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.756180 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.756982 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.758697 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.772934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1319/2664] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:40.772934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.781421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25b67-22c7-467f-905b-cc7b9fb5179f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.787265 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.787932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.788216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.788482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.789167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.791165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.816913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.818431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.828158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1346/2665] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:40.828948 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.839932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a537b8-731d-41b2-93dc-7478c1554dea service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.845856 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.846609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.846888 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.847740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.848474 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.850153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.862067 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1347/2666] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:40.865440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.881945 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1320/2667] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:40.882192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1c47b-3eec-49ed-a461-1d70019bdae1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.883399 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:40.928262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:40.959395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401a03-c79c-4a2a-bc6f-992136919442 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:40.964709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.969712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:40.969872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.969872 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.970094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:40.970193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] PATH_INFO: `/v3/projects/45f9aad3e06343ed999c216808f1d556/users/15ab0e4a3c9b49cdbb2e1ac0b64f1853/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.970282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:40.970282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:40.970282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.970282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:40.985897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f9aad3e06343ed999c216808f1d556, user_id=15ab0e4a3c9b49cdbb2e1ac0b64f1853, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:40.988972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.077848 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1321/2668] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:40 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:41.082172 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.092774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb95a-5180-4e7d-992e-3de12b45218a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.099851 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.100113 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:41.100682 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.101036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.101387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.102294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.105822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d08e64-784d-465d-be5e-d8f6874ca68c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.114173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1348/2669] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:40 2022] PUT /identity/v3/projects/45f9aad3e06343ed999c216808f1d556/users/15ab0e4a3c9b49cdbb2e1ac0b64f1853/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:41.121421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1322/2670] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:41.125175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.126683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.126976 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.127253 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.158138 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] MFA Rules not processed for user `15ab0e4a3c9b49cdbb2e1ac0b64f1853`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:41.159254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:41.224103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bfbf1-f671-48d2-b045-0e6aa524f133 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.230979 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.231680 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.231971 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.232235 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.233345 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.235500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.254744 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1349/2671] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:41.259076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.276786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af37f9-a81e-4483-ad9a-adf085dc1ef4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.283465 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.284157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1323/2672] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:41.286814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.286814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.286814 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] PATH_INFO: `/v3/projects/99d08cee51d546a2a89c85c39374e6d6/users/eceb340ed80f4722bf2e6f92aa7f60d2/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.289436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.307473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d3ab7-c5c7-4312-b7e4-1ed74f156ab2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.309783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d08cee51d546a2a89c85c39374e6d6, user_id=eceb340ed80f4722bf2e6f92aa7f60d2, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.313336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f99aff-72ab-4abf-9a3e-918fd6c4cbab admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.313891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.314775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.315132 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.315473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.316761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.319227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.432597 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1350/2673] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:41 2022] PUT /identity/v3/projects/99d08cee51d546a2a89c85c39374e6d6/users/eceb340ed80f4722bf2e6f92aa7f60d2/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:41.446061 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1324/2674] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:41.446280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.446280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.446280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.448111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.499423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] MFA Rules not processed for user `eceb340ed80f4722bf2e6f92aa7f60d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:41.500920 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:41.522657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e95a1-2b9e-4e2e-b2bc-a93201c6f190 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.527689 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.528369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.528661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.528930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.529625 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.531603 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.566116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:41.572943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1351/2675] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:41.577447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.589613 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05cbe1-4f17-481a-b9c4-f558093c17d8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.596482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.596482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.596482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.596482 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.597044 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.598968 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.621757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1325/2676] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:41.626309 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.638402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1a07c-b613-4ae7-8908-c64183931ee5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.649035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.649035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.649035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.649035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.649371 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.649728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.662096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1326/2677] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:41.665854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.677749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95601c79-38c9-4b98-bbcf-464ee940577c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.684482 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.685348 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.685700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.685871 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] PATH_INFO: `/v3/projects/810ddb60f2d347f291464c0cec8e8a3e/users/4858b524aefe4054aeeebf5e6877c17c/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.703347 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=810ddb60f2d347f291464c0cec8e8a3e, user_id=4858b524aefe4054aeeebf5e6877c17c, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.706896 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.707018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1352/2678] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:41.710647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.783560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88391e55-dc41-4e9c-aad2-ee06fba991f6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.789310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.792619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.792619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.792619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.792619 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.803192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.822564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1327/2679] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:41 2022] PUT /identity/v3/projects/810ddb60f2d347f291464c0cec8e8a3e/users/4858b524aefe4054aeeebf5e6877c17c/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:41.831144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.923293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1353/2680] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:41 2022] POST /identity/v3/users => generated 303 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:41.924488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.939521 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2652d17-461d-47be-8e06-b4b980745b2b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.942658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96748-8778-4a3e-8506-d61a437897cd admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.946501 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.947714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.948255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.948679 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.950230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.952110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.954613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.957141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:41.957461 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:41.958033 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:41.959106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:41.961795 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:41.965536 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1354/2681] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:41.970842 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:41.983393 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c51d0-2308-49d7-8df3-f949fa4f8295 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:41.995365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:41.995583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1328/2682] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:42.000661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.000661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.000661 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] PATH_INFO: `/v3/projects/37d527576c9d410a965421a7585477eb/users/f00ae955a8b6475cb2a38b6840672384/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.004713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.020787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d527576c9d410a965421a7585477eb, user_id=f00ae955a8b6475cb2a38b6840672384, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.035228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.035228 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:42.035709 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868880-9ba3-44f8-b92e-ce00a446168f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.037375 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.038383 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.038686 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.038978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] PATH_INFO: `/v3/projects/810ddb60f2d347f291464c0cec8e8a3e/users/4858b524aefe4054aeeebf5e6877c17c/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.044335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810ddb60f2d347f291464c0cec8e8a3e, user_id=4858b524aefe4054aeeebf5e6877c17c, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.047742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6079c3-91a4-44cf-b7dc-a027bd8599e0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.066219 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1329/2683] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:42 2022] PUT /identity/v3/projects/810ddb60f2d347f291464c0cec8e8a3e/users/4858b524aefe4054aeeebf5e6877c17c/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:42.071888 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.072182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.072457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.114592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1355/2684] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:41 2022] PUT /identity/v3/projects/37d527576c9d410a965421a7585477eb/users/f00ae955a8b6475cb2a38b6840672384/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:42.118795 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.123714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] MFA Rules not processed for user `4858b524aefe4054aeeebf5e6877c17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:42.125016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:42.174158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:42.220535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ded7-ce1b-412b-8ab8-f04558e5ea26 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.232738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.234152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.234440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.234717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.238621 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.240958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.250498 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1330/2685] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:42.262170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1356/2686] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:42.263660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.269610 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.274929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:42.277665 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19196a-7eee-4f62-bef1-94321b039bca None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.287193 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.288345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.288796 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.289254 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] PATH_INFO: `/v3/projects/37d527576c9d410a965421a7585477eb/users/f00ae955a8b6475cb2a38b6840672384/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.291436 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d527576c9d410a965421a7585477eb, user_id=f00ae955a8b6475cb2a38b6840672384, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.294356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d36ef-aee9-411a-b041-3a04687188a5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.318118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1331/2687] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:42 2022] PUT /identity/v3/projects/37d527576c9d410a965421a7585477eb/users/f00ae955a8b6475cb2a38b6840672384/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:42.321006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.321492 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.322211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.348588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902f013-1db9-4bb6-b295-0ced929fda87 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.371043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] MFA Rules not processed for user `f00ae955a8b6475cb2a38b6840672384`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:42.372453 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:42.431537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.432240 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.432709 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.432829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.485497 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1332/2688] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:30:42.500780 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.505602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:42.527287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:42.532110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.535071 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.540388 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8a062-9ef6-4895-b231-ab21f621f108 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.548902 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1357/2689] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:30:42.550874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.551652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.551877 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.552155 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.565029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.569874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.588194 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1333/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:42.656308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:42.667336 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29e53a-e0d4-42ca-82e0-4f4ee7d83487 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:42.679276 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:42.683234 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:42.683526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:42.683804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:42.721066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:42.774544 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:42.776450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:42.787005 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1358/2691] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:43.997804 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:44.010798 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6669f-af27-4b8f-a93e-ca8dccaab177 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:44.020167 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:44.020937 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:44.022499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:44.022499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:44.024176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:44.055775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:44.113753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:44.115380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830838b1-bd76-4724-8971-007717f529b1 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:44.130864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1334/2692] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:48.290408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:48.387464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7554f23-7de7-4bf2-ac67-81c917a7ce55 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:48.393988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:48.395947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:48.396662 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:48.396995 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] PATH_INFO: `/v3/users/eceb340ed80f4722bf2e6f92aa7f60d2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:48.398714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eceb340ed80f4722bf2e6f92aa7f60d2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:48.400701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:48.593331 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eceb340ed80f4722bf2e6f92aa7f60d2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:48.593914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eceb340ed80f4722bf2e6f92aa7f60d2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:48.594820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eceb340ed80f4722bf2e6f92aa7f60d2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:48.608766 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eceb340ed80f4722bf2e6f92aa7f60d2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:48.614035 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:48.616876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eceb340ed80f4722bf2e6f92aa7f60d2'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:48.675703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1359/2693] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:48 2022] DELETE /identity/v3/users/eceb340ed80f4722bf2e6f92aa7f60d2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:48.680552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:48.681110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:48.681213 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:48.699341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:48.700514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:48.761352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1335/2694] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:48.771623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:48.773029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:48.858043 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1c6fd-b1ca-4eed-849a-0df70ba2417b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:48.859068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:48.860291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:48.860291 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:48.860589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:48.871685 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:48.873390 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:48.885086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1360/2695] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:49.463336 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:49.464915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:49.470059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:49.471803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:49.478622 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d3875-07f6-46e4-947a-3cdd2f6cb3e6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:49.494088 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:49.495552 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:49.496146 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:49.496716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] PATH_INFO: `/v3/projects/99d08cee51d546a2a89c85c39374e6d6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:49.507642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d08cee51d546a2a89c85c39374e6d6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:49.510501 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:49.565095 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045afbab-d574-4f73-8d11-cf1ebb481497 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:49.569513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:49.570284 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:49.570638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:49.570983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:49.573317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:49.637753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:49.639196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:49.649618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1361/2696] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:30:49.685878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d08cee51d546a2a89c85c39374e6d6', 'request_id': 'req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:49.783156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d08cee51d546a2a89c85c39374e6d6', 'request_id': 'req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:49.789685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d08cee51d546a2a89c85c39374e6d6', 'request_id': 'req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:49.801067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] The token cache is being invalidate because project 99d08cee51d546a2a89c85c39374e6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:49.801357 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:49.802106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:49.803721 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1336/2697] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:49 2022] DELETE /identity/v3/projects/99d08cee51d546a2a89c85c39374e6d6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:49.807612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:49.824029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5726b-7d85-4a13-b021-e731cba93769 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:49.871821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:49.887718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:49.891702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:49.891702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:49.891702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] PATH_INFO: `/v3/users/f00ae955a8b6475cb2a38b6840672384` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:49.891702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00ae955a8b6475cb2a38b6840672384)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:49.892461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:50.077195 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00ae955a8b6475cb2a38b6840672384'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.078133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00ae955a8b6475cb2a38b6840672384'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.078889 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00ae955a8b6475cb2a38b6840672384'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.091787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00ae955a8b6475cb2a38b6840672384'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.098718 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5dbb5a6b-854c-46c1-97b4-efdb5620ba67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00ae955a8b6475cb2a38b6840672384'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.194870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1362/2698] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:49 2022] DELETE /identity/v3/users/f00ae955a8b6475cb2a38b6840672384 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:50.630456 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:50.632044 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:50.646100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ee56-3a1d-4fca-9b17-fe0a04b65169 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:50.692333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:50.710451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:50.710451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:50.710451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:50.710451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] PATH_INFO: `/v3/projects/37d527576c9d410a965421a7585477eb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:50.710861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d527576c9d410a965421a7585477eb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:50.712635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:50.806419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d527576c9d410a965421a7585477eb', 'request_id': 'req-f3e06bf3-dba3-4ca7-9919-4855ec95a553'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.897450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d527576c9d410a965421a7585477eb', 'request_id': 'req-f3e06bf3-dba3-4ca7-9919-4855ec95a553'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.907744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d527576c9d410a965421a7585477eb', 'request_id': 'req-f3e06bf3-dba3-4ca7-9919-4855ec95a553'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.923412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] The token cache is being invalidate because project 37d527576c9d410a965421a7585477eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:50.923834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.924750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f3e06bf3-dba3-4ca7-9919-4855ec95a553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:50.926606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1337/2699] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:50 2022] DELETE /identity/v3/projects/37d527576c9d410a965421a7585477eb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:50.931716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:50.934886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:50.934886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:50.951678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:50.953134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:51.027271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1363/2700] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:51.036299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.036589 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.036870 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.055228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:51.056448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:51.074169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1338/2701] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:51.077475 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.088321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a040b-d0b1-405f-b0a4-7b68c7d3c2e3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.094081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.097928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.097928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.097928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.097928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:51.097928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.099643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.112926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1364/2702] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:30:51.117640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.127031 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129de61-0817-48e8-a131-c3deac75c91e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.132377 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.133047 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.133330 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.133809 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.134557 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.136564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.261919 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1339/2703] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:51 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:51.264702 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.275848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59de40-db7f-4b66-a5d3-c408d4b839ce admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.328727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:51.339900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.340587 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.340870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.341148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.341881 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.343925 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.401355 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1365/2704] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:51 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:51.407355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.418701 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb60c0-1e11-48b7-8f0e-0b4d841c9c49 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.425936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.426702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.427179 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.427480 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.428010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.429674 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.443894 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1340/2705] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:51.445985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.458912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af7eb4-f6de-4400-8d7d-44f292c37cb2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.463460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.463460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.463460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.463460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.464175 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.465788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aae35c-df01-4473-9954-877bd6bc45f9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.477425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1366/2706] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:51.481758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.494049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bb7b7-e8eb-44cb-8b9d-1c6c502cdbdf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.498861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.499776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.500063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.500337 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] PATH_INFO: `/v3/projects/94a3b3f29eed4cf19457b36ce4bdd4d7/users/beb32ddaaafe41b7bee73af1e71ef36c/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.517016 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a3b3f29eed4cf19457b36ce4bdd4d7, user_id=beb32ddaaafe41b7bee73af1e71ef36c, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.519723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.595320 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1341/2707] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:51 2022] PUT /identity/v3/projects/94a3b3f29eed4cf19457b36ce4bdd4d7/users/beb32ddaaafe41b7bee73af1e71ef36c/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:51.606203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ae8a0-4110-4dc4-83a0-53777b5485e3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.606604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ae8a0-4110-4dc4-83a0-53777b5485e3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.606949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ae8a0-4110-4dc4-83a0-53777b5485e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.635367 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-694ae8a0-4110-4dc4-83a0-53777b5485e3 None None] MFA Rules not processed for user `beb32ddaaafe41b7bee73af1e71ef36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:51.636699 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-694ae8a0-4110-4dc4-83a0-53777b5485e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:51.706152 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1367/2708] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:51.717640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:51.720994 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:51.779263 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271ebc-3115-4500-b19d-c46898078e44 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:51.783240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:51.783892 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:51.784863 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:51.785143 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:51.794066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:51.795611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:51.805243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1342/2709] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:53.275660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af54d3-2215-4435-8cc6-943bc90982b4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:53.276186 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af54d3-2215-4435-8cc6-943bc90982b4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:53.276273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af54d3-2215-4435-8cc6-943bc90982b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:53.295463 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-40af54d3-2215-4435-8cc6-943bc90982b4 None None] MFA Rules not processed for user `ae021e81bd724b8085a2e384576946dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:53.296697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-40af54d3-2215-4435-8cc6-943bc90982b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:53.362687 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1368/2710] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:53.764785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:53.766211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:53.780484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:53.839875 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60d0d-5596-4ffb-bd66-27abbd6a2af8 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:53.844810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:53.845700 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:53.845978 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:53.846281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] PATH_INFO: `/v3/users/eaa374a7f70342e592bcfb57a4ef02bb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:53.847654 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa374a7f70342e592bcfb57a4ef02bb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:53.849583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:54.043345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa374a7f70342e592bcfb57a4ef02bb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.043345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa374a7f70342e592bcfb57a4ef02bb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.043345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa374a7f70342e592bcfb57a4ef02bb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.054581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa374a7f70342e592bcfb57a4ef02bb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.062723 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa374a7f70342e592bcfb57a4ef02bb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.145310 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1343/2711] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:53 2022] DELETE /identity/v3/users/eaa374a7f70342e592bcfb57a4ef02bb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:30:54.151494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:54.151934 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:54.152282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:54.170877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:54.172171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:54.233231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1369/2712] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:54.243483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:54.244885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:54.303408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0513f-d6c8-4fb0-a4a8-0c835cc07055 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:54.307202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:54.307853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:54.308136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:54.308417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:54.317852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:54.319308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:54.329106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1344/2713] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:54.755779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:54.757730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:54.766681 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:54.770998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7767a4b-b3fb-48e7-984d-0d0f2dc7a60e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:54.775615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:54.776538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:54.776821 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:54.777094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] PATH_INFO: `/v3/projects/a52c6692a9924a71830ccaeda5019148` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:54.778321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52c6692a9924a71830ccaeda5019148)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:54.780377 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:54.908382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52c6692a9924a71830ccaeda5019148', 'request_id': 'req-6c5167fb-a194-463a-8ce3-27b6efe85ce0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.970600 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52c6692a9924a71830ccaeda5019148', 'request_id': 'req-6c5167fb-a194-463a-8ce3-27b6efe85ce0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.977323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52c6692a9924a71830ccaeda5019148', 'request_id': 'req-6c5167fb-a194-463a-8ce3-27b6efe85ce0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.988015 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] The token cache is being invalidate because project a52c6692a9924a71830ccaeda5019148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:54.988303 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.988941 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:54.990652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1370/2714] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:54 2022] DELETE /identity/v3/projects/a52c6692a9924a71830ccaeda5019148 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:54.994758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:55.005105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6450b-2f46-4206-91ef-e53591cb609f service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:55.036893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:55.062408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:55.063300 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:55.063586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:55.063865 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] PATH_INFO: `/v3/users/15ab0e4a3c9b49cdbb2e1ac0b64f1853` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:55.065051 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ab0e4a3c9b49cdbb2e1ac0b64f1853)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:55.067559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:55.343657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ab0e4a3c9b49cdbb2e1ac0b64f1853'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:55.344239 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ab0e4a3c9b49cdbb2e1ac0b64f1853'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:55.344845 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ab0e4a3c9b49cdbb2e1ac0b64f1853'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:55.356736 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ab0e4a3c9b49cdbb2e1ac0b64f1853'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:55.364565 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ab0e4a3c9b49cdbb2e1ac0b64f1853'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:55.427856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1345/2715] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:54 2022] DELETE /identity/v3/users/15ab0e4a3c9b49cdbb2e1ac0b64f1853 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:55.830182 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:55.832133 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:55.845007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5167fb-a194-463a-8ce3-27b6efe85ce0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:55.901347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:55.902261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:55.902550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:55.902830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] PATH_INFO: `/v3/projects/45f9aad3e06343ed999c216808f1d556` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:55.904009 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f9aad3e06343ed999c216808f1d556)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:55.905921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:56.004643 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f9aad3e06343ed999c216808f1d556', 'request_id': 'req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.073614 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f9aad3e06343ed999c216808f1d556', 'request_id': 'req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.080350 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f9aad3e06343ed999c216808f1d556', 'request_id': 'req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.091664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] The token cache is being invalidate because project 45f9aad3e06343ed999c216808f1d556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:56.091943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.092591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.094126 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1371/2716] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:55 2022] DELETE /identity/v3/projects/45f9aad3e06343ed999c216808f1d556 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:56.098657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:56.100106 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:56.112459 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97725-2892-4d23-9090-8f02cb734065 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:56.167953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:56.168830 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:56.169117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:56.169393 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] PATH_INFO: `/v3/users/4858b524aefe4054aeeebf5e6877c17c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:56.170652 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4858b524aefe4054aeeebf5e6877c17c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:56.172561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:56.327757 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4858b524aefe4054aeeebf5e6877c17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.328406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4858b524aefe4054aeeebf5e6877c17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.329006 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4858b524aefe4054aeeebf5e6877c17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.336645 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:56.341847 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4858b524aefe4054aeeebf5e6877c17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.348511 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5affd66-5d9d-487d-bd28-bfd23b511c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4858b524aefe4054aeeebf5e6877c17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:56.450345 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1346/2717] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:30:56 2022] DELETE /identity/v3/users/4858b524aefe4054aeeebf5e6877c17c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:56.852589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:56.854171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:56.869947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6f45-9dc7-4a4e-b08a-e9171d8750c1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:56.889716 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:56.935696 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:56.936573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:56.936850 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:56.937120 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] PATH_INFO: `/v3/projects/810ddb60f2d347f291464c0cec8e8a3e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:56.938434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=810ddb60f2d347f291464c0cec8e8a3e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:56.940937 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.077782 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810ddb60f2d347f291464c0cec8e8a3e', 'request_id': 'req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:57.166150 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810ddb60f2d347f291464c0cec8e8a3e', 'request_id': 'req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:57.173189 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810ddb60f2d347f291464c0cec8e8a3e', 'request_id': 'req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:57.184449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] The token cache is being invalidate because project 810ddb60f2d347f291464c0cec8e8a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:30:57.184818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:57.185461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1c642e4f-1b0e-4db3-a461-05cee5f3c5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:30:57.187038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1372/2718] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:30:56 2022] DELETE /identity/v3/projects/810ddb60f2d347f291464c0cec8e8a3e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:57.192929 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.193435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.193538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.212653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:57.215934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:57.277702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1347/2719] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:57.287100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.287442 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.288227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.309723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:57.311058 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:57.320484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1373/2720] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:57.324628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.336746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dbbc8-0d8c-47b0-b374-1dc0c7e4572a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.341917 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.342821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.343108 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.343500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.344110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:30:57.344421 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.346043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.363009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1348/2721] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:30:57.364727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.374886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c5f23-a8fe-4c04-99e8-0bfa3bbfcab6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.381006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.382216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.382816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.383258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.384098 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.385990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.545527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1374/2722] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/projects => generated 399 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:57.548997 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.559296 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b78b1-235d-4619-b441-561466cd0799 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.615986 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.616653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.616933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.617206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.617893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.619832 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.697214 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1349/2723] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:30:57.701584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.712529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962bd93-e3d4-4232-81ed-f7cd39ab34b3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.717252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.717913 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.718238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.718518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.719238 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.720833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.731818 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1375/2724] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:57.735817 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.745591 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66b201-147e-4353-b648-dff4b58bf68d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.750154 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.752140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.752140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.752140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.752140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.754158 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c547d6-33e1-4428-8067-c97e53ca64e8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.763843 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1350/2725] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:57.767741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:57.778171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14fdf5-3e16-42ee-8e3a-6f8ba4a26ab0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:57.782861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:57.783732 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.784030 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.784305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] PATH_INFO: `/v3/projects/e082136773c04d25ada7cfee70b71612/users/57d2b820d96b411eabd982d4eab9f943/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.798703 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e082136773c04d25ada7cfee70b71612, user_id=57d2b820d96b411eabd982d4eab9f943, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:57.801327 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:57.868647 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1376/2726] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:57 2022] PUT /identity/v3/projects/e082136773c04d25ada7cfee70b71612/users/57d2b820d96b411eabd982d4eab9f943/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:57.880289 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:57.880608 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:57.880883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:57.909452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] MFA Rules not processed for user `57d2b820d96b411eabd982d4eab9f943`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:57.910676 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:57.939173 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:57.975070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1351/2727] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:57.979431 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.025045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:58.040705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5e11b-1e71-4499-ba4b-71e7dcd25aee admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.045374 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.050221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.050221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.050221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.050221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.050221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.163181 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1377/2728] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:30:57 2022] POST /identity/v3/projects => generated 401 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:58.167256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.179885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d059b4c-fdda-449f-9ecc-1e8b5d06681d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.246258 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.246949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.247220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.247495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.248205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.250233 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.360722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1352/2729] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:30:58 2022] POST /identity/v3/users => generated 307 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:30:58.365196 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.375451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc600aae-ac6c-4182-814b-e8584d45e761 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.380704 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.381495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.381731 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.386984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.386984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.386984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.395877 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1378/2730] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:30:58.400208 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.410510 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ccd4-31ed-48db-a5a4-4d3916e1aa39 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.416464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.417360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.417643 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.417938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] PATH_INFO: `/v3/projects/0ba007214a3a4a96858e2b17779ed653/users/c58e52457336411ca6b77219943d4c97/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.434281 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba007214a3a4a96858e2b17779ed653, user_id=c58e52457336411ca6b77219943d4c97, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.437037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.583020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1353/2731] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:58 2022] PUT /identity/v3/projects/0ba007214a3a4a96858e2b17779ed653/users/c58e52457336411ca6b77219943d4c97/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:58.587147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.597916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025661e-1e0d-4b30-a970-c6d016389fd7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.657034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.658347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.658633 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.659444 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.660176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.661859 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c395-34db-4d4a-a2b9-221042f424e7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.673665 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1379/2732] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:30:58.681039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.691684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07ffa-f780-410e-b693-1912c5bd09ea admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.696761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.697653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.697942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.698251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] PATH_INFO: `/v3/projects/0ba007214a3a4a96858e2b17779ed653/users/c58e52457336411ca6b77219943d4c97/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.700267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba007214a3a4a96858e2b17779ed653, user_id=c58e52457336411ca6b77219943d4c97, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.703035 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.714211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1354/2733] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:30:58 2022] PUT /identity/v3/projects/0ba007214a3a4a96858e2b17779ed653/users/c58e52457336411ca6b77219943d4c97/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:30:58.726346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.726645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.726931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.753658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] MFA Rules not processed for user `c58e52457336411ca6b77219943d4c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:30:58.754961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:58.820692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1380/2734] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:30:58.832913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:58.834776 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:30:58.898630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeee31-ad5b-48e8-9de0-e3b14103e8a1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:58.902342 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:58.903002 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:58.903287 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:58.903562 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:58.915598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:58.917543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:58.927764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1355/2735] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:30:59.108125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:30:59.117988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a77dd-59d6-4854-952e-a567129cdf91 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:30:59.121917 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:30:59.122612 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:30:59.125370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:30:59.125370 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:30:59.160848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:30:59.181076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:30:59.181076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0a2e-f056-4216-b211-38a6286dad65 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:30:59.189694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1381/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:30:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:06.196756 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:06.358649 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:06.394810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba805-8132-462a-ac01-7c43d152bfcc service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:06.399753 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:06.400614 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:06.400891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:06.401160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] PATH_INFO: `/v3/users/57d2b820d96b411eabd982d4eab9f943` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:06.402370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d2b820d96b411eabd982d4eab9f943)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:06.404306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:06.567112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d2b820d96b411eabd982d4eab9f943'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:06.567767 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d2b820d96b411eabd982d4eab9f943'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:06.568409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d2b820d96b411eabd982d4eab9f943'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:06.580450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d2b820d96b411eabd982d4eab9f943'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:06.587201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d2b820d96b411eabd982d4eab9f943'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:06.667607 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1356/2737] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:06 2022] DELETE /identity/v3/users/57d2b820d96b411eabd982d4eab9f943 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:06.672251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:06.672783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:06.672783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:06.690853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:06.692075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:06.778361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1382/2738] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:06.779466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:06.780966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:06.849429 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:06.851246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:06.872655 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41925-de33-4908-83df-e9a9b383e367 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:06.877969 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:06.878791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:06.879063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:06.879382 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:06.890110 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:06.891547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:06.902105 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1357/2739] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:06.957677 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677cad-e31f-4095-9222-2ad4b1e9be0b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:06.963659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:06.964454 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:06.964737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:06.965061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:06.967522 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:06.983540 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:06.983930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:06.997187 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1383/2740] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:07.385053 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:07.386626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:07.401137 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c27240-b12c-469f-be55-34d71d6b85e3 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:07.405953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:07.406913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:07.407269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:07.407612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] PATH_INFO: `/v3/projects/e082136773c04d25ada7cfee70b71612` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:07.409564 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e082136773c04d25ada7cfee70b71612)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:07.411574 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:07.508794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:07.514693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e082136773c04d25ada7cfee70b71612', 'request_id': 'req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.570942 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e082136773c04d25ada7cfee70b71612', 'request_id': 'req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.577242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e082136773c04d25ada7cfee70b71612', 'request_id': 'req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.590362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] The token cache is being invalidate because project e082136773c04d25ada7cfee70b71612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:07.590651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.591306 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.593220 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1358/2741] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:07 2022] DELETE /identity/v3/projects/e082136773c04d25ada7cfee70b71612 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:07.597243 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:07.607926 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb594f2-1791-4286-a1ad-cee38e3c6bb4 service placement] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:07.670576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:07.671490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:07.671776 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:07.672057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] PATH_INFO: `/v3/users/c58e52457336411ca6b77219943d4c97` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:07.673305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58e52457336411ca6b77219943d4c97)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:07.675372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:07.686094 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:07.887281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58e52457336411ca6b77219943d4c97'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.888725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58e52457336411ca6b77219943d4c97'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.888725 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58e52457336411ca6b77219943d4c97'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.900046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58e52457336411ca6b77219943d4c97'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.906857 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-33d69078-e26a-443f-9d13-72634aca7a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58e52457336411ca6b77219943d4c97'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:07.976178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1384/2742] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:07 2022] DELETE /identity/v3/users/c58e52457336411ca6b77219943d4c97 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:08.418794 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:08.423100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:08.434190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9ad23-ab70-4bdc-a469-14469edf4ed2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:08.492918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:08.493858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:08.494188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:08.494471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] PATH_INFO: `/v3/projects/0ba007214a3a4a96858e2b17779ed653` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:08.495620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba007214a3a4a96858e2b17779ed653)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:08.497508 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:08.594930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba007214a3a4a96858e2b17779ed653', 'request_id': 'req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:08.651014 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba007214a3a4a96858e2b17779ed653', 'request_id': 'req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:08.656710 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba007214a3a4a96858e2b17779ed653', 'request_id': 'req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:08.668368 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] The token cache is being invalidate because project 0ba007214a3a4a96858e2b17779ed653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:08.668567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:08.669249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d3e1fd09-6399-45b1-8d10-84d3d3f83d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:08.670825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1359/2743] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:08 2022] DELETE /identity/v3/projects/0ba007214a3a4a96858e2b17779ed653 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:08.675891 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:08.676349 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:08.676446 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:08.694684 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:08.695929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:08.757010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1385/2744] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:08.765771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:08.766085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:08.766362 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:08.785397 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:08.786616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:08.794602 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1360/2745] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:08.798870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:08.808695 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff93d0-2455-4323-96ef-08c046a8d875 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:08.813391 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:08.814171 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:08.814538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:08.814897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:08.815692 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:08.816070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:08.817764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:08.830051 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1386/2746] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:31:08.834788 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:08.844057 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752fcbb-ce83-4afb-a958-40ad3e5805be None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:08.849548 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:08.850693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:08.850693 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:08.851046 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:08.851504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:08.853384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:08.938299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1361/2747] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:08 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:08.942271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:08.952566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b1c2d-0473-47c9-82f5-b8a094794200 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.008358 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.009027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.009313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.009591 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.010354 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.012258 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.085723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1387/2748] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:08 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:09.089786 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.100093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec7d2-067d-4a9a-bb71-7de885747e69 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.104791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.105446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.105728 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.106001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.106813 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.108384 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.118861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1362/2749] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:09.122779 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.129604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:09.133334 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1680e-1484-4246-85d7-f9146856eaa1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.138083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.138741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.139017 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.139301 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.140070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.141666 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91da3c-b44f-49cc-b98f-3000f1345543 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.154062 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1388/2750] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:09.156677 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.166596 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8317a0-8b06-4738-a95e-0b27111f7407 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.171136 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.172036 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.172316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.172590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] PATH_INFO: `/v3/projects/8ae6290549df493aa45c197bcefaa7ad/users/384510e371bd4c6a8315dde42256305f/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.188151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae6290549df493aa45c197bcefaa7ad, user_id=384510e371bd4c6a8315dde42256305f, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.191537 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.195027 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:09.269327 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1363/2751] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:09 2022] PUT /identity/v3/projects/8ae6290549df493aa45c197bcefaa7ad/users/384510e371bd4c6a8315dde42256305f/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:09.273906 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21938519-3a67-40f2-9728-2be30a58967c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.274226 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21938519-3a67-40f2-9728-2be30a58967c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.274579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21938519-3a67-40f2-9728-2be30a58967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.302753 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-21938519-3a67-40f2-9728-2be30a58967c None None] MFA Rules not processed for user `384510e371bd4c6a8315dde42256305f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:09.304054 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-21938519-3a67-40f2-9728-2be30a58967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:09.369928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1389/2752] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:09.374452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.434955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0601db1-b328-4df1-9a28-1da2957527ff admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.439644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.440782 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.441066 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.441341 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.442096 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.443975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.520799 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1364/2753] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:09 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:09.524884 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21938519-3a67-40f2-9728-2be30a58967c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.535494 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21938519-3a67-40f2-9728-2be30a58967c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.591050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.591720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.592003 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.592279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.592959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.594909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.696623 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1390/2754] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:09 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:09.700488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.710838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddcdc-9f13-4b43-b831-b65f4fbc5a0f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.715500 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.716156 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.716442 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.716717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.717438 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.719107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.729821 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1365/2755] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:09.733909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.744620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3d358-849e-4b13-b124-10235ec3b55f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.749379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.750287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.750574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.750854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] PATH_INFO: `/v3/projects/0177e508734a41dcbf0d2ad527c91f70/users/405887fa25364e999825e8375cf2fe60/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.765392 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0177e508734a41dcbf0d2ad527c91f70, user_id=405887fa25364e999825e8375cf2fe60, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.768110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.839461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1391/2756] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:09 2022] PUT /identity/v3/projects/0177e508734a41dcbf0d2ad527c91f70/users/405887fa25364e999825e8375cf2fe60/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:09.843521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.854227 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4ce30-3a2d-45bf-931c-5d49db50c1db admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.911802 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.912478 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.912844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.913120 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.913858 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.915454 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ef8b6-ba88-40e0-9301-d983e16dc8fa admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.927279 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1366/2757] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:09.931581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:09.942323 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bfd8-b31f-4661-ab1e-829dcc96e89e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:09.947592 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:09.948422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:09.948801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:09.949085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] PATH_INFO: `/v3/projects/0177e508734a41dcbf0d2ad527c91f70/users/405887fa25364e999825e8375cf2fe60/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:09.951117 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0177e508734a41dcbf0d2ad527c91f70, user_id=405887fa25364e999825e8375cf2fe60, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:09.953798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:09.966727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1392/2758] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:09 2022] PUT /identity/v3/projects/0177e508734a41dcbf0d2ad527c91f70/users/405887fa25364e999825e8375cf2fe60/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:10.035853 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:10.036359 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:10.036495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:10.063761 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] MFA Rules not processed for user `405887fa25364e999825e8375cf2fe60`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:10.065005 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:10.116583 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:10.133773 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1367/2759] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:10.144799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:10.146505 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:10.187573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:10.204786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ea88f-2339-4deb-9731-b506dc8b00fb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:10.208893 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:10.209549 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:10.209876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:10.210205 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:10.219451 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:10.221024 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:10.231398 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1393/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:10.336521 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:10.346904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13464-e76a-407a-8c6d-95ca8c21d9b7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:10.350540 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:10.351190 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:10.351471 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:10.351743 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:10.407749 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:10.409943 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeabe5-d99a-4ed8-8218-372b94bd9ea1 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:10.419122 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1368/2761] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:13.606245 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:13.680279 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3dc43-784f-4a1c-9a6c-96f98df498d1 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:13.685077 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:13.685961 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:13.686290 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:13.686560 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] PATH_INFO: `/v3/users/384510e371bd4c6a8315dde42256305f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:13.687894 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=384510e371bd4c6a8315dde42256305f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:13.689930 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:13.924852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384510e371bd4c6a8315dde42256305f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:13.925517 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384510e371bd4c6a8315dde42256305f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:13.926158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384510e371bd4c6a8315dde42256305f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:13.938566 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384510e371bd4c6a8315dde42256305f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:13.945931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384510e371bd4c6a8315dde42256305f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.023518 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1394/2762] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:13 2022] DELETE /identity/v3/users/384510e371bd4c6a8315dde42256305f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:14.028823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:14.029452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:14.029744 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:14.050606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:14.055657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:14.113017 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1369/2763] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:14.122953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:14.124609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:14.184006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5349-50f5-41ca-8a2b-6eb9202091cc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:14.188004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:14.188669 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:14.189108 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:14.189670 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:14.199569 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:14.201086 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:14.207018 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:14.212875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1395/2764] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:14.687150 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:14.688741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:14.702169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe68bf4-b0b0-4300-a0ba-1471faf6d1ae None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:14.706168 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:14.707073 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:14.707387 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:14.707667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] PATH_INFO: `/v3/projects/8ae6290549df493aa45c197bcefaa7ad` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:14.708889 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae6290549df493aa45c197bcefaa7ad)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:14.710844 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:14.839087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae6290549df493aa45c197bcefaa7ad', 'request_id': 'req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.907960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae6290549df493aa45c197bcefaa7ad', 'request_id': 'req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.913852 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae6290549df493aa45c197bcefaa7ad', 'request_id': 'req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.925183 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] The token cache is being invalidate because project 8ae6290549df493aa45c197bcefaa7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:14.925468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.926121 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:14.927651 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1370/2765] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:14 2022] DELETE /identity/v3/projects/8ae6290549df493aa45c197bcefaa7ad => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:14.931804 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:14.942212 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d93e97-1efa-4cf4-a572-b4ecd64ac056 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:14.997883 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:14.998717 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:14.998999 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:14.999274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] PATH_INFO: `/v3/users/405887fa25364e999825e8375cf2fe60` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:15.000480 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405887fa25364e999825e8375cf2fe60)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:15.002512 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:15.255673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405887fa25364e999825e8375cf2fe60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:15.256247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405887fa25364e999825e8375cf2fe60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:15.256969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405887fa25364e999825e8375cf2fe60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:15.268741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405887fa25364e999825e8375cf2fe60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:15.275532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405887fa25364e999825e8375cf2fe60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:15.366174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1396/2766] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:14 2022] DELETE /identity/v3/users/405887fa25364e999825e8375cf2fe60 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:15.835640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:15.836918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:15.851335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238be1-d3e3-4bbe-97ec-f2e67ca6aa02 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:15.883944 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:15.953884 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:15.954174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:15.958634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:15.958634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] PATH_INFO: `/v3/projects/0177e508734a41dcbf0d2ad527c91f70` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:15.958634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0177e508734a41dcbf0d2ad527c91f70)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:15.962323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.004730 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.006412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.020848 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df491-faf6-4961-af30-879e4eadbd86 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.025515 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.026423 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.026711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.026992 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] PATH_INFO: `/v3/users/beb32ddaaafe41b7bee73af1e71ef36c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.028202 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb32ddaaafe41b7bee73af1e71ef36c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.030293 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.114996 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0177e508734a41dcbf0d2ad527c91f70', 'request_id': 'req-e9c5ad5b-821f-4ca2-b6f5-836905876b59'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.176962 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0177e508734a41dcbf0d2ad527c91f70', 'request_id': 'req-e9c5ad5b-821f-4ca2-b6f5-836905876b59'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.183272 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0177e508734a41dcbf0d2ad527c91f70', 'request_id': 'req-e9c5ad5b-821f-4ca2-b6f5-836905876b59'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.195094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] The token cache is being invalidate because project 0177e508734a41dcbf0d2ad527c91f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:16.195408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.196294 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e9c5ad5b-821f-4ca2-b6f5-836905876b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.197829 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1371/2767] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:15 2022] DELETE /identity/v3/projects/0177e508734a41dcbf0d2ad527c91f70 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:16.202930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd41f-0bb8-41b1-9433-baab52d802e1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.203216 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd41f-0bb8-41b1-9433-baab52d802e1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.203494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd41f-0bb8-41b1-9433-baab52d802e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.229758 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb32ddaaafe41b7bee73af1e71ef36c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.230432 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb32ddaaafe41b7bee73af1e71ef36c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.231031 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb32ddaaafe41b7bee73af1e71ef36c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.244473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b80fd41f-0bb8-41b1-9433-baab52d802e1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:16.245441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb32ddaaafe41b7bee73af1e71ef36c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.245840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b80fd41f-0bb8-41b1-9433-baab52d802e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.253701 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-daf32be6-41ab-43fd-936a-b97a085f24d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb32ddaaafe41b7bee73af1e71ef36c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:16.256361 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:16.306838 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1372/2768] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:16.314355 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.315202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.315475 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.331339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1397/2769] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:16 2022] DELETE /identity/v3/users/beb32ddaaafe41b7bee73af1e71ef36c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:16.332864 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:16.334111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.336010 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.336450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.336745 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.357455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:16.358688 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.375238 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:16.394406 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1373/2770] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:16.398751 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.407664 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d74753-de52-4ada-b788-6e7f67d880e1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.414624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.415429 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.415731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.415999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.416752 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:16.417317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.418981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.421287 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1398/2771] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:16.431449 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.432807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.439097 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1374/2772] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:31:16.440684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.450731 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab77367-adc3-476c-ae0d-8e16841128e7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.455505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.456165 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.456443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.456714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.457380 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.459303 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.490088 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1da9-be7c-4bff-aa94-2d996efa0e50 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.493865 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.494625 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.496164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.496164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.511247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.513041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.523375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1399/2773] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:16.556283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1375/2774] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:16.557437 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.567061 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ff107-50fe-43ce-9199-b463cd5abbd2 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.634302 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.637147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.637147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.637147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.637147 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.638148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.753039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1400/2775] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:16.757076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.766205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a3c3-36a7-4e97-a8ca-becb4b02579f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.771447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.772151 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.772434 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.772712 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.773477 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.775152 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.786514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1376/2776] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:16.791028 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.802083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c58761-1fe8-4c87-8af1-670b5a0fe151 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.806915 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.807588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.809022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.809022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.809022 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.810604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955250a6-c5bb-42ed-b350-8ff6f05c5a26 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.821483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1401/2777] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:16.825775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.836600 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609206c-7c9b-45fa-8790-d7f5eb133f56 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:16.841369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:16.842282 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.842629 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.842914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] PATH_INFO: `/v3/projects/a2c8d83ed4df4058b627051ef663a7c0/users/500548972bf04d38b95aa3e6463abc4f/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.858274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c8d83ed4df4058b627051ef663a7c0, user_id=500548972bf04d38b95aa3e6463abc4f, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:16.860925 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:16.960411 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1377/2778] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:16 2022] PUT /identity/v3/projects/a2c8d83ed4df4058b627051ef663a7c0/users/500548972bf04d38b95aa3e6463abc4f/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:16.972736 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:16.973029 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:16.973308 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:16.994494 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:16.996648 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:16.999004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:17.002130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] MFA Rules not processed for user `500548972bf04d38b95aa3e6463abc4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:17.003526 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:17.066369 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed715-8345-4f2a-a314-fa18bc0448a6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.071581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.076262 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1402/2779] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:17.076705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.076705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.076705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] PATH_INFO: `/v3/projects/94a3b3f29eed4cf19457b36ce4bdd4d7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.076705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a3b3f29eed4cf19457b36ce4bdd4d7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.076705 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.077222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.090734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a08a9e-9245-4645-8080-c68505054ae9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.097551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.098282 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.098576 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.098862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.099574 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.102045 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.207741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a3b3f29eed4cf19457b36ce4bdd4d7', 'request_id': 'req-6cbd967c-e741-49a3-8e97-8c273d535d2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:17.232815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1403/2780] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:17.236537 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.288931 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a3b3f29eed4cf19457b36ce4bdd4d7', 'request_id': 'req-6cbd967c-e741-49a3-8e97-8c273d535d2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:17.298597 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a3b3f29eed4cf19457b36ce4bdd4d7', 'request_id': 'req-6cbd967c-e741-49a3-8e97-8c273d535d2f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:17.306720 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043fe5a-a704-46a3-ac7a-34a0dba62c2f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.311620 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.312272 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.312404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] The token cache is being invalidate because project 94a3b3f29eed4cf19457b36ce4bdd4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:17.312555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.312659 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:17.312834 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.313319 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cbd967c-e741-49a3-8e97-8c273d535d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:17.313523 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.315467 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1378/2781] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:16 2022] DELETE /identity/v3/projects/94a3b3f29eed4cf19457b36ce4bdd4d7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:17.317179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.325739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd0c3-097f-40e1-ad36-0d5fdc29d81b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.326331 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd0c3-097f-40e1-ad36-0d5fdc29d81b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.326610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bd0c3-097f-40e1-ad36-0d5fdc29d81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.334460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:17.345277 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ff9bd0c3-097f-40e1-ad36-0d5fdc29d81b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:17.350722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ff9bd0c3-097f-40e1-ad36-0d5fdc29d81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:17.354588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1379/2782] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:17.362115 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.362404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.362681 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.380298 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:17.381470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:17.389297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1380/2783] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:17.393856 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.402634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d96b7-1919-4014-88d1-43b1a0c2956b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.407262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.407918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.408206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.408481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.409192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:17.409499 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.411224 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.419046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1404/2784] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:17.422366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.423680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1381/2785] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 20:31:17.424650 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:17.427822 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.436799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806b80-b2f7-4784-bade-42c4e58faae5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.437116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cce39-48ea-40bc-b3f2-c1b598a1ea5f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.442049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.442203 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.442741 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.442875 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.443021 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.443131 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.443293 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.443404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.443966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.444068 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.445636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.445828 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.459723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1405/2786] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:17.462485 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.472580 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c087c56-81ac-42f4-95b2-48db3c48ef6d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.477275 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.478038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.482740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.482740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.482740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.482740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.491364 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1406/2787] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:17.495417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.505221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2dfb73-3d23-4c2a-9928-3931f829c3bb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.509916 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.511447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.511447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.511447 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] PATH_INFO: `/v3/projects/555581da74d4468f938c46fd570b1484/users/27c1943a5ab04c1ba910d0c81ded97ac/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.525952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555581da74d4468f938c46fd570b1484, user_id=27c1943a5ab04c1ba910d0c81ded97ac, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.528936 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8294b6-95ba-45dd-a598-2b7bd14c4568 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.543559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1382/2788] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:17.547543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.607043 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d14e2-981d-4658-a616-1dabf4243089 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.612273 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.612957 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.613356 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.613484 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.614448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.616269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.622123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1407/2789] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:17 2022] PUT /identity/v3/projects/555581da74d4468f938c46fd570b1484/users/27c1943a5ab04c1ba910d0c81ded97ac/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:17.643477 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.643765 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.644146 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.669658 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] MFA Rules not processed for user `27c1943a5ab04c1ba910d0c81ded97ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:17.670904 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:17.683321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:17.708846 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1383/2790] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:17.713072 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.722444 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6970cd-72a8-4ea0-a837-b06966183c98 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.736100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1408/2791] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:17.744714 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.794412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.795176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.795462 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.795732 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.796457 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.798052 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.809119 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1384/2792] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:17.817439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6563f-06cc-4fb5-9867-0adf7146dd55 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.818615 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.819719 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.820389 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.820674 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.821217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.821952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.824317 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.828874 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d582-6176-4a76-addb-343fd99da40d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.834269 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.834933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.838644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.838644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.838644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.838644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.847820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1385/2793] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:17.852039 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.861820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535b276-fe66-4423-94ca-bc5c197f109e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:17.866370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:17.867199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.867474 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.867742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] PATH_INFO: `/v3/projects/c992951d34c0405796ff56319b746efb/users/a3a44a4faf6e480088a39a0a3f420538/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:17.882556 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c992951d34c0405796ff56319b746efb, user_id=a3a44a4faf6e480088a39a0a3f420538, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:17.885199 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef77b-39b6-4b8c-b062-7cccad9f786e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:17.950100 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1409/2794] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:17.956012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:17.962635 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1386/2795] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:17 2022] PUT /identity/v3/projects/c992951d34c0405796ff56319b746efb/users/a3a44a4faf6e480088a39a0a3f420538/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:17.976425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:17.976425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:17.976425 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.013192 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:18.021198 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0e1a-158d-47e8-ae1d-caf95ee37ae9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.028805 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] MFA Rules not processed for user `a3a44a4faf6e480088a39a0a3f420538`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:18.030063 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.066738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:18.080366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.081176 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.081516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.081853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.082579 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.084531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.108446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1387/2796] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:18.119184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.120549 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.175710 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1410/2797] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:17 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:18.177199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.181582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ceca2-bad8-47b6-93ea-eb8bef17e100 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.185441 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.186148 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.186440 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.186811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.187177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058eae-c984-4417-838c-f608510a7ba5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.191912 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.192565 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.192853 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.193134 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.200050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.200170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.200269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.200376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.205593 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1411/2798] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:18.206573 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1388/2799] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:18.209909 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.224096 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272964c-aeed-464d-96db-7c1aad00ee98 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.229405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.229778 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.230092 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.230471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] PATH_INFO: `/v3/projects/cbd108d42c2e4ebb89cbc60a56594729/users/c30dd00e3dbe4c21913c887fb7e793c3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.237240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.247359 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd108d42c2e4ebb89cbc60a56594729, user_id=c30dd00e3dbe4c21913c887fb7e793c3, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.250274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.255618 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1e2ad-3a71-48f8-92af-0197afaded85 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.257401 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.258403 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.258754 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.259103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] PATH_INFO: `/v3/users/a3a44a4faf6e480088a39a0a3f420538` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.260435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a44a4faf6e480088a39a0a3f420538)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.262458 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.358946 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1412/2800] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:18 2022] PUT /identity/v3/projects/cbd108d42c2e4ebb89cbc60a56594729/users/c30dd00e3dbe4c21913c887fb7e793c3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:18.363210 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.434249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e274775-7653-4989-848a-d717d8a8ca40 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.436307 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.436974 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.437248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.437532 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.438326 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.439932 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.453756 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1413/2801] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 20:31:18.459983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.467357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca905c-0dd4-4bc7-abef-d604f4884cf2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.473193 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.474770 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.475059 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.475347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] PATH_INFO: `/v3/projects/cbd108d42c2e4ebb89cbc60a56594729/users/c30dd00e3dbe4c21913c887fb7e793c3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.478763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd108d42c2e4ebb89cbc60a56594729, user_id=c30dd00e3dbe4c21913c887fb7e793c3, role_id=e9cc8b774de34cf99d5bb6930f6ce1c4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.480702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a44a4faf6e480088a39a0a3f420538'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:18.481835 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a44a4faf6e480088a39a0a3f420538'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:18.481969 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610b1d5-675b-4cf4-8353-43ab739648e9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.483111 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a44a4faf6e480088a39a0a3f420538'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:18.493040 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1414/2802] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:18 2022] PUT /identity/v3/projects/cbd108d42c2e4ebb89cbc60a56594729/users/c30dd00e3dbe4c21913c887fb7e793c3/roles/e9cc8b774de34cf99d5bb6930f6ce1c4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:18.500191 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a44a4faf6e480088a39a0a3f420538'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:18.502929 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.505497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.505497 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.508616 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8f523d53-e98b-407a-b5ae-13e22a594f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a44a4faf6e480088a39a0a3f420538'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:18.539963 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] MFA Rules not processed for user `c30dd00e3dbe4c21913c887fb7e793c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:18.541173 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.605251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1389/2803] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:18 2022] DELETE /identity/v3/users/a3a44a4faf6e480088a39a0a3f420538 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:18.609634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.615642 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.615738 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1415/2804] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:18.615823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.625535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.627313 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.632409 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:18.633670 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.639203 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:18.640107 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:18.700357 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f9c13-c62f-47ea-b27c-9a9e2f984e65 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.712366 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1390/2805] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:18.722485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.725207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:18.755880 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.756460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.756796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.757072 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.784536 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6196ac-1af0-41ae-8d45-766e10dc555c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:18.789770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:18.790588 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:18.791318 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:18.791606 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:18.802505 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.803954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.813902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1391/2806] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:18.824252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:18.825700 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:18.836435 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1416/2807] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 20:31:18.990408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:18.990408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:19.002250 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6041a-cb84-4d78-aba4-b6c6e8cb7edf service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:19.007326 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:19.007904 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.008187 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.008464 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.010316 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:19.074993 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:19.076431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea5a8b-2258-4a38-8ab0-fafc14016b00 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:19.086451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1392/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:19.287739 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:19.289286 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:19.303484 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ba-c78a-4dfd-8482-2c32dd5c57f9 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:19.308387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:19.309352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.309648 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.309972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] PATH_INFO: `/v3/projects/c992951d34c0405796ff56319b746efb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.311295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c992951d34c0405796ff56319b746efb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:19.313427 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:19.468807 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c992951d34c0405796ff56319b746efb', 'request_id': 'req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:19.537232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c992951d34c0405796ff56319b746efb', 'request_id': 'req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:19.544223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c992951d34c0405796ff56319b746efb', 'request_id': 'req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:19.551091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:19.555868 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] The token cache is being invalidate because project c992951d34c0405796ff56319b746efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:19.556153 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:19.556861 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-26f6b8e3-d9aa-42d3-8cf5-4b6f1796602b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:19.558563 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1417/2809] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:19 2022] DELETE /identity/v3/projects/c992951d34c0405796ff56319b746efb => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:19.563323 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.563610 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.563885 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.583451 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:19.584644 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:19.636465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:19.651133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1393/2810] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:19.659070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.660555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.660555 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.678140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:19.680165 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:19.688769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1418/2811] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:19.693668 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:19.703261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154333a-f0c6-44fe-bc89-58768ce7c158 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:19.708469 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:19.709126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.712412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.712412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.712412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:19.712412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:19.712412 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:19.724305 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1394/2812] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:31:19.728584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:19.737876 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bc616-8f58-4206-988c-758731555b46 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:19.742734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:19.743382 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.743659 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.743928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.744616 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:19.746690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:19.836694 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1419/2813] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:19 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:19.840985 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:19.852267 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a91b1-3cfc-4f86-91eb-87fd9dcd0c96 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:19.911513 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:19.912184 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:19.912466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:19.912738 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:19.913473 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:19.915395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:20.003332 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1395/2814] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:19 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:20.004487 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:20.014757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65960de1-780a-4bf3-9346-f0dfb489ef52 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:20.019471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:20.020132 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:20.025734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:20.025734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:20.025734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:20.025734 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:20.033274 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1420/2815] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:20.037262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:20.047176 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535cf9-6bfd-4e4c-a051-9d4d6a27af59 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:20.052354 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:20.054419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:20.054419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:20.054419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:20.054419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:20.055834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc7b8-5287-422a-b589-984cb67ecf55 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:20.066360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1396/2816] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:20.070375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:20.083014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213bdc-5aee-4a4b-93a1-c456cd87e613 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:20.085845 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:20.086589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:20.086864 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:20.087139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] PATH_INFO: `/v3/projects/fba86b652dc24d4784eb3caa4d3ed300/users/47e7f40810184611bfdd184a35dc26fc/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:20.101958 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba86b652dc24d4784eb3caa4d3ed300, user_id=47e7f40810184611bfdd184a35dc26fc, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:20.104634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:20.182646 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1421/2817] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:20 2022] PUT /identity/v3/projects/fba86b652dc24d4784eb3caa4d3ed300/users/47e7f40810184611bfdd184a35dc26fc/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:20.187660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04854-1d54-460a-a47b-f55df0b23d6a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:20.187948 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04854-1d54-460a-a47b-f55df0b23d6a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:20.188252 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04854-1d54-460a-a47b-f55df0b23d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:20.206561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-afb04854-1d54-460a-a47b-f55df0b23d6a None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:20.217763 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-afb04854-1d54-460a-a47b-f55df0b23d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:20.266818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1397/2818] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:31:20.276960 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:20.278655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:20.334217 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8018a5-a135-40b0-9028-5926cf9073fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:20.337764 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:20.338441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:20.338728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:20.339008 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:20.347835 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:20.349259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:20.358486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1422/2819] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:21.560938 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283866-37fd-4237-9cf4-748436fb0f4e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:21.561417 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283866-37fd-4237-9cf4-748436fb0f4e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:21.561515 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283866-37fd-4237-9cf4-748436fb0f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:21.588117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4f283866-37fd-4237-9cf4-748436fb0f4e None None] MFA Rules not processed for user `47e7f40810184611bfdd184a35dc26fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:21.589325 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4f283866-37fd-4237-9cf4-748436fb0f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:21.636781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:21.654999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1398/2820] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:21.666386 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:21.667768 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:21.724230 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158aa32-81ed-4d6c-b906-dc554e817f3c service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:21.727828 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:21.728462 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:21.728740 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:21.729014 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:21.737082 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:21.738545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:21.747101 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1423/2821] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:23.109205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834eda8-6aa3-4b36-bcdd-5c91fe4dba7d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:23.113801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834eda8-6aa3-4b36-bcdd-5c91fe4dba7d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:23.113801 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834eda8-6aa3-4b36-bcdd-5c91fe4dba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:23.128056 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1834eda8-6aa3-4b36-bcdd-5c91fe4dba7d None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:23.129244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1834eda8-6aa3-4b36-bcdd-5c91fe4dba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:23.196493 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1399/2822] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 20:31:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:23.247513 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:23.249011 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:23.296784 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:23.326728 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd63f3-cb08-45c9-8b21-767c1e46673c service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:23.332294 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:23.332943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:23.333255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:23.333534 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:23.341491 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:23.343111 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:23.351032 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1424/2823] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:31:23.453717 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deac93-4f2b-4184-b320-0c8c3eddef6c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:23.454025 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deac93-4f2b-4184-b320-0c8c3eddef6c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:23.454322 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deac93-4f2b-4184-b320-0c8c3eddef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:23.500049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-49deac93-4f2b-4184-b320-0c8c3eddef6c None None] MFA Rules not processed for user `47e7f40810184611bfdd184a35dc26fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:23.500049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-49deac93-4f2b-4184-b320-0c8c3eddef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:23.512135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1400/2824] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:25.441721 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:25.443192 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:25.484248 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af859ff-375e-4926-a24e-51b7bcd2c1bf service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:25.489373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:25.490347 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:25.490823 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:25.491106 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] PATH_INFO: `/v3/users/500548972bf04d38b95aa3e6463abc4f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:25.492492 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500548972bf04d38b95aa3e6463abc4f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:25.494557 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:25.657261 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500548972bf04d38b95aa3e6463abc4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:25.658265 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500548972bf04d38b95aa3e6463abc4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:25.658905 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500548972bf04d38b95aa3e6463abc4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:25.672510 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500548972bf04d38b95aa3e6463abc4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:25.681900 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500548972bf04d38b95aa3e6463abc4f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:25.783602 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1425/2825] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:25 2022] DELETE /identity/v3/users/500548972bf04d38b95aa3e6463abc4f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:25.788241 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:25.789170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:25.789455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:25.811093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:25.814699 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:25.905975 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1401/2826] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:25.916907 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:25.918375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:26.017943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71613810-639e-4376-bb32-076cd17bd764 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:26.021755 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:26.022787 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:26.023255 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:26.023706 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:26.038422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:26.039816 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:26.061188 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1426/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:26.562404 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:26.564295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:26.577490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730b267-0ce7-424e-af47-3564104f2458 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:26.583437 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:26.584450 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:26.584797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:26.585140 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] PATH_INFO: `/v3/projects/a2c8d83ed4df4058b627051ef663a7c0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:26.586529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c8d83ed4df4058b627051ef663a7c0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:26.588685 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:26.716295 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c8d83ed4df4058b627051ef663a7c0', 'request_id': 'req-fbdbb5e2-38cb-4009-b7de-3050926ed520'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:26.775762 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c8d83ed4df4058b627051ef663a7c0', 'request_id': 'req-fbdbb5e2-38cb-4009-b7de-3050926ed520'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:26.782370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c8d83ed4df4058b627051ef663a7c0', 'request_id': 'req-fbdbb5e2-38cb-4009-b7de-3050926ed520'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:26.794598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] The token cache is being invalidate because project a2c8d83ed4df4058b627051ef663a7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:26.795256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:26.795771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:26.807087 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1402/2828] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:26 2022] DELETE /identity/v3/projects/a2c8d83ed4df4058b627051ef663a7c0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:26.810933 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:26.842495 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0a70-405f-4d71-9f8b-08cf136b1b82 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:26.896788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:26.947216 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:26.948177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:26.948410 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:26.948792 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] PATH_INFO: `/v3/users/27c1943a5ab04c1ba910d0c81ded97ac` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:26.955794 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c1943a5ab04c1ba910d0c81ded97ac)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:26.957858 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:27.229966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c1943a5ab04c1ba910d0c81ded97ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:27.230660 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c1943a5ab04c1ba910d0c81ded97ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:27.231269 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c1943a5ab04c1ba910d0c81ded97ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:27.253280 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c1943a5ab04c1ba910d0c81ded97ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:27.277140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c1943a5ab04c1ba910d0c81ded97ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:27.367503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1427/2829] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:26 2022] DELETE /identity/v3/users/27c1943a5ab04c1ba910d0c81ded97ac => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:27.899636 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:27.903098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:27.917211 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbb5e2-38cb-4009-b7de-3050926ed520 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:27.936529 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:28.023117 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:28.024204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:28.024770 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:28.025313 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] PATH_INFO: `/v3/projects/555581da74d4468f938c46fd570b1484` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:28.026970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555581da74d4468f938c46fd570b1484)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:28.029328 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:28.143170 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555581da74d4468f938c46fd570b1484', 'request_id': 'req-14664631-02cb-4224-ad59-758841b270c8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.205640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555581da74d4468f938c46fd570b1484', 'request_id': 'req-14664631-02cb-4224-ad59-758841b270c8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.226837 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555581da74d4468f938c46fd570b1484', 'request_id': 'req-14664631-02cb-4224-ad59-758841b270c8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.242694 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] The token cache is being invalidate because project 555581da74d4468f938c46fd570b1484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:28.243038 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.243619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.245609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1403/2830] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:27 2022] DELETE /identity/v3/projects/555581da74d4468f938c46fd570b1484 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:28.249352 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:28.250860 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:28.273479 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3725-7f13-4516-b75b-9447e197675f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:28.366200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:28.366200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:28.366452 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:28.367075 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] PATH_INFO: `/v3/users/c30dd00e3dbe4c21913c887fb7e793c3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:28.368199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30dd00e3dbe4c21913c887fb7e793c3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:28.370407 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:28.555266 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30dd00e3dbe4c21913c887fb7e793c3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.556019 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30dd00e3dbe4c21913c887fb7e793c3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.556741 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30dd00e3dbe4c21913c887fb7e793c3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.569434 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30dd00e3dbe4c21913c887fb7e793c3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.571767 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:28.586089 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0a3adceb-8726-420b-a88c-4ce4d9d80e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30dd00e3dbe4c21913c887fb7e793c3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:28.677231 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1428/2831] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:28 2022] DELETE /identity/v3/users/c30dd00e3dbe4c21913c887fb7e793c3 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:31:29.347126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:29.348861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:29.376542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14664631-02cb-4224-ad59-758841b270c8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:29.463914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:29.465672 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:29.466174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:29.466680 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] PATH_INFO: `/v3/projects/cbd108d42c2e4ebb89cbc60a56594729` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:29.468365 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd108d42c2e4ebb89cbc60a56594729)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:29.470446 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:29.770432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:29.774118 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd108d42c2e4ebb89cbc60a56594729', 'request_id': 'req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:29.901435 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd108d42c2e4ebb89cbc60a56594729', 'request_id': 'req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:29.908202 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd108d42c2e4ebb89cbc60a56594729', 'request_id': 'req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:29.919862 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] The token cache is being invalidate because project cbd108d42c2e4ebb89cbc60a56594729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:29.920149 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:29.920823 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c8ef7aa1-2fa4-4f65-81ee-ffa802470630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:29.922376 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1404/2832] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:29 2022] DELETE /identity/v3/projects/cbd108d42c2e4ebb89cbc60a56594729 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:29.928433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:29.928966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:29.929083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:29.948603 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:29.949799 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:30.038069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1429/2833] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:30.044270 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.044559 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.044842 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.067029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:30.068339 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:30.077019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1405/2834] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:30.082783 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.095711 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ced196-076c-4d7b-9d77-0d1cad5437b6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.103278 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.103942 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.104224 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.104500 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.105227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:30.105531 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.107166 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.125083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1430/2835] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:31:30.126335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.137878 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb98d79-681f-4de8-b524-46ba3f87b0e8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.142750 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.143431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.143713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.143988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.144683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.147980 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.241419 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1406/2836] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:30 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:30.245914 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.263644 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab79cfa-d0a2-4d18-ba58-1bb0b5f2ac8c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.341122 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.342177 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.342561 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.342972 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.343998 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.346206 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.435607 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1431/2837] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:30 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:30.447646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.459653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe30c20-ab10-4bfa-bff7-deef4c8f9ed4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.464879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.465620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.465915 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.466223 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.466954 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.468547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.480249 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1407/2838] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:30.483928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.494820 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4275-5bc0-45fb-9580-735b7a54e1b5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.500066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.500746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.501023 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.501295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.502046 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.504360 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075ff82-66ca-49b5-881d-4d86f421a4f5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.518793 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1432/2839] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:30.520160 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.531274 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f39a08-a979-44c6-bf17-c35dd3239f2e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.536432 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.537349 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.537658 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.537955 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] PATH_INFO: `/v3/projects/d2ca99728e3443de8b564415903aff15/users/a4f61654970d4b7686ea2c51218bd467/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.554169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ca99728e3443de8b564415903aff15, user_id=a4f61654970d4b7686ea2c51218bd467, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.557081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.645370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1408/2840] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:30 2022] PUT /identity/v3/projects/d2ca99728e3443de8b564415903aff15/users/a4f61654970d4b7686ea2c51218bd467/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:30.647164 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226ecc-494b-4911-a45a-87738cb52ffc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.647461 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226ecc-494b-4911-a45a-87738cb52ffc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.647803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226ecc-494b-4911-a45a-87738cb52ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.677887 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-77226ecc-494b-4911-a45a-87738cb52ffc None None] MFA Rules not processed for user `a4f61654970d4b7686ea2c51218bd467`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:30.679746 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-77226ecc-494b-4911-a45a-87738cb52ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:30.719863 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:30.749584 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1433/2841] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:30.761726 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:30.763684 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:30.830008 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01c73d-c1c6-4dd7-b7a3-55bb90d68a5e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:30.833923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:30.834707 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:30.835059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:30.835427 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:30.844779 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:30.846918 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:30.858307 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1409/2842] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:40.816405 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=10 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:40.852567 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:47.149786 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024200-bc49-452e-885b-43e519701451 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:47.150319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024200-bc49-452e-885b-43e519701451 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:47.150425 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024200-bc49-452e-885b-43e519701451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:47.168433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e024200-bc49-452e-885b-43e519701451 None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:47.169712 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e024200-bc49-452e-885b-43e519701451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:47.226093 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1434/2843] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:31:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:47.241443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:47.241898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:47.310538 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4f57d-4480-4ee9-bed0-c872325729e9 service glance] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:47.315413 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:47.316055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:47.316338 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:47.316613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:47.324797 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:47.326420 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:47.334739 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1410/2844] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:31:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:31:50.215664 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e024200-bc49-452e-885b-43e519701451 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:50.217249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e024200-bc49-452e-885b-43e519701451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:50.289041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e024200-bc49-452e-885b-43e519701451 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:50.294055 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:50.294737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:50.296529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:50.296529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:50.297225 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:50.309486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:50.311246 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:50.321957 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1435/2845] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:31:50.354879 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:50.356506 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:50.429758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492382a-0708-479e-af7f-83d9bd100e52 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:50.434901 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:50.435898 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:50.436262 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:50.436576 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] PATH_INFO: `/v3/users/ae021e81bd724b8085a2e384576946dc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:50.437820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae021e81bd724b8085a2e384576946dc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:50.439785 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:50.578913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae021e81bd724b8085a2e384576946dc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:50.579490 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae021e81bd724b8085a2e384576946dc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:50.580098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae021e81bd724b8085a2e384576946dc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:50.593481 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae021e81bd724b8085a2e384576946dc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:50.601029 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-696fba25-f7b7-4428-9872-d47df58144a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae021e81bd724b8085a2e384576946dc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:50.651174 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1411/2846] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:31:50 2022] DELETE /identity/v3/users/ae021e81bd724b8085a2e384576946dc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:50.887074 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:50.898406 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77650db0-c31e-4606-bd82-db508b52d97e service placement] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:50.964083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:50.964983 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:50.965273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:50.965550 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] PATH_INFO: `/v3/projects/a656c669853f430a9a23e5568d1adb7a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:50.966852 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a656c669853f430a9a23e5568d1adb7a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:50.968789 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.072538 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:51.081455 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a656c669853f430a9a23e5568d1adb7a', 'request_id': 'req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:51.169325 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a656c669853f430a9a23e5568d1adb7a', 'request_id': 'req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:51.176142 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a656c669853f430a9a23e5568d1adb7a', 'request_id': 'req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:51.187439 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] The token cache is being invalidate because project a656c669853f430a9a23e5568d1adb7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:31:51.187727 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:51.188388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4068f7c1-c0a1-4391-9f77-13fcfb24cdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:31:51.189919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1436/2847] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:31:50 2022] DELETE /identity/v3/projects/a656c669853f430a9a23e5568d1adb7a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:51.195522 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.195819 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.196103 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.214818 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:51.216028 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:51.220660 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:31:51.275995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1412/2848] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:51.283945 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.284440 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.284551 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.303125 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:51.304341 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:51.312400 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1437/2849] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:51.316758 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.327081 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48255-7dfc-4bd7-bc3b-25398a7d1e08 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.331613 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.332284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.332566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.332854 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.333667 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:31:51.333922 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.335566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.347260 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1413/2850] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:31:51.352214 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.362788 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d259e5e-48d1-4c05-bba5-7fcfc4eef5d4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.368379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.373013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.373013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.373013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.373013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.373013 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.452259 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1438/2851] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:31:51 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:31:51.457715 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.467566 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c15eb-5b00-4086-b48d-cd253d0d7f18 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.525586 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.526297 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.526582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.526859 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.527561 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.529468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.645611 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1414/2852] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:31:51 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 20:31:51.649559 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.660007 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9c730-a61d-47f9-a468-91e82f8132ba admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.665141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.665796 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.666140 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.666422 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.667158 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.668769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.680486 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1439/2853] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:51.684448 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.694960 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd776c-0755-4d34-9a15-a55027064324 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.699916 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.700587 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.700963 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.701243 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.702299 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.704093 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20f914-9cdd-47d9-9b8b-e5260388716a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.714914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1415/2854] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:31:51.724387 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.729911 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7870e1-f68b-4274-b748-97997c387a0e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:51.734708 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:51.735754 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.736079 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.736366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] PATH_INFO: `/v3/projects/1cc7a6629aca4ff5a9a57ebab23f76bf/users/126343ff805347928220db52a6995ec1/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.752673 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc7a6629aca4ff5a9a57ebab23f76bf, user_id=126343ff805347928220db52a6995ec1, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:51.755529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:51.816618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1440/2855] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:31:51 2022] PUT /identity/v3/projects/1cc7a6629aca4ff5a9a57ebab23f76bf/users/126343ff805347928220db52a6995ec1/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:31:51.880488 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:51.880771 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:51.881049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:51.909803 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] MFA Rules not processed for user `126343ff805347928220db52a6995ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:31:51.911581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:51.976503 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1416/2856] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:31:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:31:51.987722 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:31:51.990081 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:31:52.047737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253821-82ce-42de-8a86-054f23bd9e94 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:31:52.051504 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef73ca7ffe94b79b085dcfa1eae278d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:31:52.052156 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:31:52.052441 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:31:52.052715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:31:52.061199 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:31:52.062697 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434abddb-8396-4750-aa84-da74f1374d56 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:31:52.072675 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1441/2857] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:31:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:32:01.972543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:02.067956 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:15.497060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:15.498547 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:15.561876 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668ecfb-3c14-4ad9-ac99-84ef8e04fb05 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:15.566390 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:15.567257 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:15.567598 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:15.567882 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] PATH_INFO: `/v3/users/a4f61654970d4b7686ea2c51218bd467` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:15.569067 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f61654970d4b7686ea2c51218bd467)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:15.571261 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:15.776514 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f61654970d4b7686ea2c51218bd467'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:15.777815 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f61654970d4b7686ea2c51218bd467'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:15.778468 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f61654970d4b7686ea2c51218bd467'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:15.790504 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f61654970d4b7686ea2c51218bd467'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:15.797433 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f61654970d4b7686ea2c51218bd467'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:15.878650 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1417/2858] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:32:15 2022] DELETE /identity/v3/users/a4f61654970d4b7686ea2c51218bd467 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:32:15.883144 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:15.887222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:15.887222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:15.901527 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:32:15.902757 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:15.960252 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1442/2859] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:32:15.969787 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:15.971204 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:16.027095 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886488-68e9-42fb-9d32-686d22b544e7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:16.030764 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:16.031408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:16.031691 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:16.031966 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:16.041075 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:16.043201 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:16.051711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1418/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:32:16.442070 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:16.443589 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:16.463039 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179313db-9ebf-4d2e-a276-7d9b379cba0d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:16.479263 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:16.480986 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:16.483556 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:16.486145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] PATH_INFO: `/v3/projects/d2ca99728e3443de8b564415903aff15` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:16.486145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ca99728e3443de8b564415903aff15)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:16.494723 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:16.600148 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ca99728e3443de8b564415903aff15', 'request_id': 'req-32d41ddb-56f6-459a-ae56-d446be372223'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:16.667588 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ca99728e3443de8b564415903aff15', 'request_id': 'req-32d41ddb-56f6-459a-ae56-d446be372223'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:16.675006 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ca99728e3443de8b564415903aff15', 'request_id': 'req-32d41ddb-56f6-459a-ae56-d446be372223'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:16.685456 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] The token cache is being invalidate because project d2ca99728e3443de8b564415903aff15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:32:16.685829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:16.686562 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-32d41ddb-56f6-459a-ae56-d446be372223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:16.688229 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1443/2861] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:32:16 2022] DELETE /identity/v3/projects/d2ca99728e3443de8b564415903aff15 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:32:26.047637 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:26.682233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:45.889065 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:45.890569 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:45.903304 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabc93-852f-40de-8944-ecd3612a4449 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:45.958335 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:45.959255 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:45.959542 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:45.959820 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] PATH_INFO: `/v3/users/126343ff805347928220db52a6995ec1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:45.961071 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126343ff805347928220db52a6995ec1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:45.963049 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:46.154718 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126343ff805347928220db52a6995ec1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.155364 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126343ff805347928220db52a6995ec1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.155965 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126343ff805347928220db52a6995ec1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.168166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126343ff805347928220db52a6995ec1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.174926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126343ff805347928220db52a6995ec1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.249924 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1419/2862] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:32:45 2022] DELETE /identity/v3/users/126343ff805347928220db52a6995ec1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:32:46.254471 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:46.254959 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:46.255076 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:46.272749 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:32:46.273935 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:46.330615 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1444/2863] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:32:46.340810 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:46.342256 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:46.405742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0598432-1785-4ef4-ac66-015d2a8cdd61 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:46.409353 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:46.410009 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:46.410321 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:46.410601 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:46.419883 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:46.421317 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:46.431116 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1420/2864] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:32:46.805170 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:46.806693 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:46.819656 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f999df-d422-4d94-b1dc-4580bdfedf94 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:46.825169 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:46.826178 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:46.826404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:46.826678 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] PATH_INFO: `/v3/projects/1cc7a6629aca4ff5a9a57ebab23f76bf` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:46.827878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc7a6629aca4ff5a9a57ebab23f76bf)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:46.829803 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:46.930141 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc7a6629aca4ff5a9a57ebab23f76bf', 'request_id': 'req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.992545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc7a6629aca4ff5a9a57ebab23f76bf', 'request_id': 'req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:46.999404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc7a6629aca4ff5a9a57ebab23f76bf', 'request_id': 'req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:47.010083 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] The token cache is being invalidate because project 1cc7a6629aca4ff5a9a57ebab23f76bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:32:47.010365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:47.011000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:32:47.012618 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1445/2865] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:32:46 2022] DELETE /identity/v3/projects/1cc7a6629aca4ff5a9a57ebab23f76bf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:32:56.426620 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:57.006949 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:32:58.287612 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:32:58.289010 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:58.299932 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051dee9-14dc-49f9-b5d5-9b5963693abc service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:32:58.347811 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:32:58.348447 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:32:58.348729 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:32:58.349007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:32:58.350714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:32:58.418912 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:32:58.420370 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec82cdb-6496-44a9-9c28-4a90548c980c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:32:58.430995 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1421/2866] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:32:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:03.303837 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:03.305288 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:03.317445 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2eed0-b5ea-48dd-bc6d-49c02879cc41 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:03.321300 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:03.321951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:03.322268 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:03.322581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:03.324310 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:03.337990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:03.339919 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:03.356636 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1446/2867] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:08.426259 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=54 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:13.346321 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:18.506455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:18.506903 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:18.507011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:18.525020 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] MFA Rules not processed for user `f42e373971c649ea89f8764d6cf142dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:33:18.526237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:18.590109 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1422/2868] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:33:18.716773 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:18.718951 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:18.800897 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c33806-9a43-4edd-b07d-3d98964e9b29 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:18.807233 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc42255281f442bb146fe386f432cc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:18.808012 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:18.808295 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:18.808568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:18.817004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:18.819174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:18.827063 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1447/2869] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 20:33:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 20:33:20.753900 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:20.764657 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0153-d9f5-4e43-9551-6c0169ea01fa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:20.769085 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:20.769840 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:20.775930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:20.775930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:20.775930 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:20.836913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:20.838581 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:20.848999 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1423/2870] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:28.758885 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:28.760379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:28.775227 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0e6b6-9cbf-4a44-86f2-4af4f5d13e67 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:28.779135 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:28.780004 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:28.780412 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:28.780748 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] PATH_INFO: `/v3/users/e6dc0d72c1fb4c2fb4f0b704ffafadb1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:28.781921 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6dc0d72c1fb4c2fb4f0b704ffafadb1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:28.783871 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:28.982655 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6dc0d72c1fb4c2fb4f0b704ffafadb1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:28.983329 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6dc0d72c1fb4c2fb4f0b704ffafadb1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:28.984529 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6dc0d72c1fb4c2fb4f0b704ffafadb1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.000886 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6dc0d72c1fb4c2fb4f0b704ffafadb1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.011947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6dc0d72c1fb4c2fb4f0b704ffafadb1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.077729 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1448/2871] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:33:28 2022] DELETE /identity/v3/users/e6dc0d72c1fb4c2fb4f0b704ffafadb1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:33:29.443934 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:29.453653 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfa868-cd5d-489e-8dc1-2d72a6f3947b service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:29.533465 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:29.534415 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:29.534716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:29.534988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] PATH_INFO: `/v3/projects/add3d6328c754244a17f23d21ff30fac` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:29.536157 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add3d6328c754244a17f23d21ff30fac)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:29.538159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:29.554862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:29.557091 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:29.573139 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285fd1-9c27-44ba-95f0-2c2dd7289c6f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:29.578645 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:29.579573 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:29.579855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:29.580130 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] PATH_INFO: `/v3/users/7231b60e5f9541629fec2291c547f0f4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:29.581457 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7231b60e5f9541629fec2291c547f0f4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:29.583408 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:29.644308 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:29.650240 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add3d6328c754244a17f23d21ff30fac', 'request_id': 'req-1fb68327-9db3-43df-b76f-e9beefb6bef4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.723144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add3d6328c754244a17f23d21ff30fac', 'request_id': 'req-1fb68327-9db3-43df-b76f-e9beefb6bef4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.728958 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add3d6328c754244a17f23d21ff30fac', 'request_id': 'req-1fb68327-9db3-43df-b76f-e9beefb6bef4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.739775 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] The token cache is being invalidate because project add3d6328c754244a17f23d21ff30fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:33:29.740060 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.740687 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.742268 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1424/2872] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:33:29 2022] DELETE /identity/v3/projects/add3d6328c754244a17f23d21ff30fac => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:33:29.746264 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:29.748098 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:29.761483 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7231b60e5f9541629fec2291c547f0f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.762346 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7231b60e5f9541629fec2291c547f0f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.763109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7231b60e5f9541629fec2291c547f0f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.771034 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:29.776568 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7231b60e5f9541629fec2291c547f0f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.783467 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7231b60e5f9541629fec2291c547f0f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.810735 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb68327-9db3-43df-b76f-e9beefb6bef4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:29.816013 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:29.816921 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:29.817206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:29.817483 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] PATH_INFO: `/v3/users/241d9d242d8c4c7d91b7573d494f2d71` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:29.819054 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] RBAC: Authorizing `identity:delete_user(user_id=241d9d242d8c4c7d91b7573d494f2d71)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:29.820951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:29.849476 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1449/2873] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:33:29 2022] DELETE /identity/v3/users/7231b60e5f9541629fec2291c547f0f4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:33:29.990860 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241d9d242d8c4c7d91b7573d494f2d71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.991545 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241d9d242d8c4c7d91b7573d494f2d71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:29.992228 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241d9d242d8c4c7d91b7573d494f2d71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.019988 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241d9d242d8c4c7d91b7573d494f2d71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.027242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241d9d242d8c4c7d91b7573d494f2d71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.085366 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:30.087026 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:30.129221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1425/2874] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:33:29 2022] DELETE /identity/v3/users/241d9d242d8c4c7d91b7573d494f2d71 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:33:30.158667 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed22f0c-9439-4577-8c3c-3f6dacf179b9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:30.222763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:30.222763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:30.222763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:30.222763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] PATH_INFO: `/v3/projects/26567fb564314003bf5ae836737b990e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:30.223878 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26567fb564314003bf5ae836737b990e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:30.226103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:30.357690 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26567fb564314003bf5ae836737b990e', 'request_id': 'req-cc290809-80b5-4c9d-88e2-a471977672e6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.371385 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:30.372800 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:30.386624 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884776a-2c73-4799-9a55-6fef5214525a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:30.433076 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:30.437409 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26567fb564314003bf5ae836737b990e', 'request_id': 'req-cc290809-80b5-4c9d-88e2-a471977672e6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.444428 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26567fb564314003bf5ae836737b990e', 'request_id': 'req-cc290809-80b5-4c9d-88e2-a471977672e6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.444702 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:30.445690 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:30.445970 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:30.446284 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] PATH_INFO: `/v3/projects/6155dcead5db4331929493c1ac0ecfb9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:30.447634 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6155dcead5db4331929493c1ac0ecfb9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:30.449551 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:30.455516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] The token cache is being invalidate because project 26567fb564314003bf5ae836737b990e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:33:30.455801 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.456470 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.458050 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1450/2875] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:33:30 2022] DELETE /identity/v3/projects/26567fb564314003bf5ae836737b990e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:33:30.461990 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:30.464671 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:30.476128 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290809-80b5-4c9d-88e2-a471977672e6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:30.480634 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:30.481488 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:30.481815 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:30.482127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] PATH_INFO: `/v3/users/f97764af4bd84f97b0c4f5c8abb3a080` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:30.483339 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97764af4bd84f97b0c4f5c8abb3a080)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:30.485855 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:30.490401 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:30.553333 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6155dcead5db4331929493c1ac0ecfb9', 'request_id': 'req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.649470 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6155dcead5db4331929493c1ac0ecfb9', 'request_id': 'req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.659410 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6155dcead5db4331929493c1ac0ecfb9', 'request_id': 'req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.665715 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97764af4bd84f97b0c4f5c8abb3a080'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.666416 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97764af4bd84f97b0c4f5c8abb3a080'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.667066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97764af4bd84f97b0c4f5c8abb3a080'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.678139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] The token cache is being invalidate because project 6155dcead5db4331929493c1ac0ecfb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:33:30.678760 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.680541 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.681433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97764af4bd84f97b0c4f5c8abb3a080'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.683558 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1426/2876] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:33:30 2022] DELETE /identity/v3/projects/6155dcead5db4331929493c1ac0ecfb9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:33:30.690000 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-cc8936ee-6564-4e88-8904-d66b09a57bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97764af4bd84f97b0c4f5c8abb3a080'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:30.753223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1451/2877] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:33:30 2022] DELETE /identity/v3/users/f97764af4bd84f97b0c4f5c8abb3a080 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 20:33:31.019923 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:31.021543 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:31.086292 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e32-26d9-47a2-aa2c-fd65fa4e5e9c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:31.140941 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:31.141806 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:31.142126 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:31.142407 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] PATH_INFO: `/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:31.143570 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5e14b0b9834f4088b7b6ee89ab33c1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:31.145460 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:31.226949 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:31.242360 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5e14b0b9834f4088b7b6ee89ab33c1', 'request_id': 'req-87db779d-98cb-40e9-8bba-055304e42e06'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:31.321580 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5e14b0b9834f4088b7b6ee89ab33c1', 'request_id': 'req-87db779d-98cb-40e9-8bba-055304e42e06'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:31.327144 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5e14b0b9834f4088b7b6ee89ab33c1', 'request_id': 'req-87db779d-98cb-40e9-8bba-055304e42e06'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:31.337769 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] The token cache is being invalidate because project 5e5e14b0b9834f4088b7b6ee89ab33c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:33:31.338094 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:31.338711 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:33:31.340251 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1427/2878] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:33:31 2022] DELETE /identity/v3/projects/5e5e14b0b9834f4088b7b6ee89ab33c1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:33:34.002496 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:34.002985 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:34.003103 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:34.020328 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] MFA Rules not processed for user `47e7f40810184611bfdd184a35dc26fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:33:34.021528 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:34.075136 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1452/2879] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:33:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:33:34.703135 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:34.704577 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:34.772099 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db779d-98cb-40e9-8bba-055304e42e06 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:34.776981 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:34.777626 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:34.777905 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:34.778206 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:34.779951 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:34.791992 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:34.793572 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:34.803188 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1428/2880] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:37.573928 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:37.575651 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:37.630849 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcf8d-002a-4f24-abd4-913eb2e57a3f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:37.634379 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:37.635020 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:37.635299 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:37.635572 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:37.637305 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:37.647057 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:37.648472 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:37.657073 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1453/2881] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:38.513011 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:38.514619 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:38.589079 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43309-5ef2-4610-9f53-6e96fb57669b service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:38.595873 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:38.596590 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:38.596926 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:38.597205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:38.599007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:38.612991 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:38.614714 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:38.625746 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1429/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:45.553271 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:33:45.554829 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:45.570866 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbda3c-0fe1-4225-beac-df90d4f52c0c service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:33:45.576375 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf871584be84949a82a9da23fb16153', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:33:45.577085 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:33:45.577372 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:33:45.577652 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:33:45.579404 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:33:45.594737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:33:45.594737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:33:45.603947 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1454/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:33:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:33:48.621141 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=80,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:33:55.599433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:34:38.187742 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:38.199661 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a292-95fd-4ead-b5f8-0dc499a13332 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:38.205301 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:38.205936 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:38.206244 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:38.207030 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:38.208466 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:38.225825 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:38.227455 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:38.238139 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1430/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:34:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:34:38.305564 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:38.307038 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:38.373159 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b08b-59d1-4530-8954-3ed5d5333cfb service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:38.378516 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:38.379388 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:38.379668 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:38.379943 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] PATH_INFO: `/v3/users/47e7f40810184611bfdd184a35dc26fc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:38.381123 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e7f40810184611bfdd184a35dc26fc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:38.383251 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:38.540420 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e7f40810184611bfdd184a35dc26fc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:38.540993 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e7f40810184611bfdd184a35dc26fc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:38.541601 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e7f40810184611bfdd184a35dc26fc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:38.553966 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e7f40810184611bfdd184a35dc26fc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:38.560931 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0bef6bc8-df0c-444b-9de2-87579bdf75a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e7f40810184611bfdd184a35dc26fc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:38.630638 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1455/2885] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:34:38 2022] DELETE /identity/v3/users/47e7f40810184611bfdd184a35dc26fc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:34:38.872395 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:38.885242 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83e524-eacc-4418-aaf7-dbeff277bc8d service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:38.939205 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:38.940070 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:38.940352 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:38.940628 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] PATH_INFO: `/v3/projects/fba86b652dc24d4784eb3caa4d3ed300` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:38.941791 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba86b652dc24d4784eb3caa4d3ed300)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:38.943953 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.021872 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba86b652dc24d4784eb3caa4d3ed300', 'request_id': 'req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:39.064914 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba86b652dc24d4784eb3caa4d3ed300', 'request_id': 'req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:39.070747 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba86b652dc24d4784eb3caa4d3ed300', 'request_id': 'req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:39.081774 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] The token cache is being invalidate because project fba86b652dc24d4784eb3caa4d3ed300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:34:39.082100 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:39.082722 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5a70f4eb-3fa9-4aa1-9dfb-3824d09b1cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:39.084237 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1431/2886] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:34:38 2022] DELETE /identity/v3/projects/fba86b652dc24d4784eb3caa4d3ed300 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:34:39.089222 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.089577 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.089862 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.109110 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:34:39.110953 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:39.167157 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1456/2887] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 20:34:39.175361 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.175683 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.175961 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.194933 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:34:39.196134 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:39.204059 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1432/2888] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:34:39.208247 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.217964 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4027c02-8f23-4f02-8070-9edb5c2daec4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.223161 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.223952 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.224137 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.224413 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.225145 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 20:34:39.225450 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.227078 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.239330 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1457/2889] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 20:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 20:34:39.243646 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.252861 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9878c-fa57-46db-acc0-c615290928e9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.257716 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.258402 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.261528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.261528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.261528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.261528 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.340350 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1433/2890] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 20:34:39 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:34:39.344984 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.354737 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f1a3-2e2a-4c42-8a3a-347f72b0b683 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.358221 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:34:39.409535 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.410223 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.410503 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.410775 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.411490 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.413394 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.502273 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1458/2891] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 20:34:39 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 20:34:39.506523 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.517343 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c5ea-bd4f-4c69-b78f-f564186ec674 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.521920 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.522713 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.523001 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.523275 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.523998 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.525582 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.536485 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1434/2892] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:34:39.540249 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.550194 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d4bb0-15f7-4910-86a8-ac44e7a98b02 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.554892 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.555545 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.555830 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.556109 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.556833 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.558460 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b08dc-44be-4453-9c05-8b1b865cd435 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.568373 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1459/2893] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 20:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 20:34:39.572283 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.582479 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1569-64f5-4589-98e0-db4400e4cf67 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.587037 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.587902 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.588182 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.588452 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] PATH_INFO: `/v3/projects/9f0740623f5d4261bcd7844955ec1e86/users/b3a0c1c675314fa8bd41cf288c457016/roles/177528ba0fa94e3a9ef5327bad1d58fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.603007 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0740623f5d4261bcd7844955ec1e86, user_id=b3a0c1c675314fa8bd41cf288c457016, role_id=177528ba0fa94e3a9ef5327bad1d58fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.605631 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.670640 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1435/2894] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 20:34:39 2022] PUT /identity/v3/projects/9f0740623f5d4261bcd7844955ec1e86/users/b3a0c1c675314fa8bd41cf288c457016/roles/177528ba0fa94e3a9ef5327bad1d58fe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:34:39.681127 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.681421 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.681705 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.709027 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] MFA Rules not processed for user `b3a0c1c675314fa8bd41cf288c457016`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:34:39.710319 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:39.772042 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1460/2895] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:34:39.782443 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:39.784209 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:39.798159 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:34:39.840535 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc8109-0769-4158-ae6c-1d6fd347b4b6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:39.843968 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '567602cb67c447c798d4009f77140b85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:39.844609 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:39.844893 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:39.845169 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:39.853431 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:39.854891 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916e422-8e00-4367-91e7-424bca460e5f service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:39.864112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1436/2896] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:34:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:34:49.769174 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:34:49.922195 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=20 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:34:57.965581 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:58.027433 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9abce3-5d0c-42c8-b49b-78d9a73b69a5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:58.031988 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:58.032854 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:58.033805 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:58.034114 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] PATH_INFO: `/v3/users/b3a0c1c675314fa8bd41cf288c457016` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:58.035297 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a0c1c675314fa8bd41cf288c457016)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:58.037218 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:58.198200 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a0c1c675314fa8bd41cf288c457016'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:58.198769 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a0c1c675314fa8bd41cf288c457016'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:58.199369 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a0c1c675314fa8bd41cf288c457016'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:58.211365 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a0c1c675314fa8bd41cf288c457016'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:58.218281 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a0c1c675314fa8bd41cf288c457016'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:58.287750 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1461/2897] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 20:34:57 2022] DELETE /identity/v3/users/b3a0c1c675314fa8bd41cf288c457016 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:34:58.292312 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:58.294285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:58.294285 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:58.309834 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] MFA Rules not processed for user `a03bfabd375944c7923bb6e092aa3498`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 20:34:58.311055 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:58.368196 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1437/2898] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 20:34:58.378232 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:58.379609 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:58.434417 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c70b6-46fd-44b0-bcf4-73deef952490 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:58.438069 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '617e5a9c04ec4d5fa7833ab46264142d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:58.438707 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:58.438987 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:58.439759 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:58.448790 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:58.450179 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:58.460066 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1462/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:34:58.870381 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:34:58.871913 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:34:58.886630 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ba59d-bdb2-41db-85e2-1f3f73cd9416 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:34:58.892128 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a03bfabd375944c7923bb6e092aa3498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f5277fd37f8044ad9345367a8cd3028d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:34:58.893221 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:34:58.893509 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:34:58.893781 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] PATH_INFO: `/v3/projects/9f0740623f5d4261bcd7844955ec1e86` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:34:58.895019 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0740623f5d4261bcd7844955ec1e86)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:34:58.897166 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:34:59.006408 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0740623f5d4261bcd7844955ec1e86', 'request_id': 'req-32cfb544-3312-4847-9b94-43d3f5d9264d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:59.085133 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0740623f5d4261bcd7844955ec1e86', 'request_id': 'req-32cfb544-3312-4847-9b94-43d3f5d9264d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:59.090880 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0740623f5d4261bcd7844955ec1e86', 'request_id': 'req-32cfb544-3312-4847-9b94-43d3f5d9264d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:59.102207 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] The token cache is being invalidate because project 9f0740623f5d4261bcd7844955ec1e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 20:34:59.102495 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:59.103112 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-32cfb544-3312-4847-9b94-43d3f5d9264d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 20:34:59.104639 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1438/2900] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 20:34:58 2022] DELETE /identity/v3/projects/9f0740623f5d4261bcd7844955ec1e86 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 20:35:08.455208 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:35:09.099234 dv-cinder-843018-1-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 20:36:10.937810 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 20:36:10.939219 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:36:11.078604 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544879e5-6e59-4182-a3fe-ae90bd089b50 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 20:36:11.135800 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01927a09fd62479a9640dccdda24f762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a53c00eca8a494bba1641d08ffd89d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 20:36:11.136478 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 20:36:11.136763 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 20:36:11.137041 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 20:36:11.138870 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 20:36:11.149844 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 20:36:11.151464 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbebce-0025-4094-99d5-67dc8cc4f53e service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 20:36:11.161798 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1463/2901] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 20:36:21.157049 dv-cinder-843018-1-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}